Skip to content

Studio: preserve durations across reasoning blocks - #7520

Merged
danielhanchen merged 3 commits into
unslothai:mainfrom
oobabooga:fix/studio-reasoning-group-durations
Jul 28, 2026
Merged

Studio: preserve durations across reasoning blocks#7520
danielhanchen merged 3 commits into
unslothai:mainfrom
oobabooga:fix/studio-reasoning-group-durations

Conversation

@oobabooga

@oobabooga oobabooga commented Jul 27, 2026

Copy link
Copy Markdown
Member

Follow-up to #7444.

A tool-enabled response can contain more than one reasoning block, such as one before a tool call and another before the final answer. Studio stored only one reasoningDuration value for the whole assistant message, so every completed block displayed the duration of the final block. A response with 2 seconds of reasoning, a tool call, and 5 more seconds of reasoning therefore showed 5 seconds on both blocks.

Reproduction

  1. Load a reasoning model and enable tools.
  2. Send a prompt that makes the model reason, call a tool, then reason again.
  3. Finish the response and reload the conversation.

Before this change, both reasoning labels used the last server summary. After this change, each label keeps the duration of its own reasoning pass during streaming and after reload.

Fix

  • Track reasoning duration by visible reasoning-group index.
  • Persist a reasoningDurations array while retaining the final scalar value for older messages and consumers.
  • Keep local fallback timing aligned when a provider omits a summary, uses structured reasoning parts, or closes a reasoning block atomically.
  • Ignore summaries that have no corresponding visible reasoning block.
  • Defer summaries from a suppressed GGUF forced re-prompt until its plain answer becomes visible, and discard them when the pass remains a hidden tool call.
  • Add focused frontend duration tests to the Studio frontend CI job.

Tool execution, sampling, and model output are unchanged.

Verification

  • Live Playwright reproduction through the real Studio UI showed independent 2-second and 5-second labels.
  • The saved assistant message contained reasoningDurations: [2, 5], and both labels remained correct after a page reload without another completion request.
  • 6 focused frontend duration tests passed.
  • 78 GGUF tool-loop tests passed under Python 3.12 and Python 3.13.
  • Frontend typecheck and production build passed.
  • Ruff, Python compilation, and whitespace checks passed.
  • Focused ESLint introduced no new findings. The existing findings in the touched frontend files remain unchanged.

@oobabooga
oobabooga requested a review from danielhanchen as a code owner July 27, 2026 19:43
@danielhanchen

Copy link
Copy Markdown
Member

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: f5be56908b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Jul 28, 2026
danielhanchen added a commit to danielhanchen/unsloth-staging-2 that referenced this pull request Jul 28, 2026
A rendered reasoning group can be closed and then reopened: parseAssistantContent
coalesces adjacent reasoning parts, so a provider that emits each block as a
complete <think>...</think> chunk lands several blocks in one group. The tracker
wrote a group's duration once and never revisited it, so such a group froze at
its first close and displayed 0 seconds.

Measure from the first time an index becomes visible rather than from the last
startGroup, and reopen a closed group while its reasoning text is still growing.
Gating on growth is what stops the timer running on into the answer. A duration
supplied by the server is now recorded as authoritative so local timing cannot
overwrite it.

Also fill indices that a single delta skips. startGroup(n) could jump past
earlier indices and leave array holes, which JSON.stringify persists as null; a
skipped group became visible and closed inside the same chunk, so it gets a
measured zero instead.

Test discovery now globs tests/, so a second test file cannot be silently
skipped by CI, and tsconfig.test.json puts tests/ under typecheck for the first
time.
@danielhanchen

Copy link
Copy Markdown
Member

@codex review

danielhanchen added a commit to danielhanchen/unsloth-staging-2 that referenced this pull request Jul 28, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: be8cd0464a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danielhanchen
danielhanchen merged commit fc861cc into unslothai:main Jul 28, 2026
50 of 54 checks passed
danielhanchen added a commit to Souravrajvi0/unsloth that referenced this pull request Jul 28, 2026
Resolve the reasoning-duration collision: main (unslothai#7520) replaced the single
reasoningStartAt/reasoningDuration pair with a per-group tracker and moved
extractDeltaText into parse-assistant-content, while this branch made the
close-tag classification structural for unslothai#7066.

- extractDeltaText keeps main's structuredReasoningContinues and regains the
  unslothai#7066 neutralization plus the closeOffsets the adapter registers as known
  wrapper boundaries.
- hasUnclosedThinkTag keeps main's semantics but resolves the close
  structurally, so a quoted or fenced literal </think> no longer stops the
  timer; the adapter passes the streaming, known-close and resume options.
- The reasoning group now starts when raw reasoning ARRIVES and stays open
  while a split marker is held back, and finishGroup accepts the deferred
  close instant so an unclosed fence does not bill the answer as thought time.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants