Studio: prevent long streaming Markdown stalls - #7892
Merged
danielhanchen merged 1 commit intoAug 5, 2026
Conversation
|
Codex Review: Didn't find any major issues. You're on a roll. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
danielhanchen
force-pushed
the
fix/studio-streamdown-starvation
branch
from
August 5, 2026 14:18
06cb2d6 to
2a93676
Compare
|
Codex Review: Didn't find any major issues. Can't wait for the next one! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
danielhanchen
added a commit
that referenced
this pull request
Aug 6, 2026
#7949) Streamdown memoises each block on its content, so a block that has finished streaming is never re-parsed. With the zero-duration animated path enabled in #7892 that leaves every per-word <span data-sd-animate> wrapper, and its live CSS animation, in the DOM for the life of the message: about 2150 spans and 2150 running animations on a 14k-character answer, and 80 percent more markup than the same answer rendered without streaming. Key each block on Streamdown's isAnimating flag so every block re-parses once when the message completes, which produces exactly the markup a non-streamed message produces. The streaming fix from #7892 is unchanged. Co-authored-by: danielhanchen <unslothai@gmail.com>
danielhanchen
added a commit
that referenced
this pull request
Aug 16, 2026
The long-task total is the metric the budgets turn on, and it reads 0 both
when the render is free and when the observer never ran. observe({type:
"longtask"}) is specified to abort silently on an engine that lacks the
entry type rather than throw, so the try/catch around it never fired: under
firefox or webkit the harness scored a perfect zero and exited 0. Detect
support with PerformanceObserver.supportedEntryTypes, record it, and fail
the run when no long tasks were seen or when throttling was disabled.
Also:
- add the entry to tsconfig.app.json, which lists the smoke entries one by
one, so npm run typecheck actually covers the 270 lines it was reported
against
- write the JSON under logs/ like every sibling harness instead of dropping
an untracked stream-pacing.json in the repo root, and create the directory
- treat an exported-but-empty SMOKE_BASE_URL as unset, matching the siblings;
it drove "" as the base URL and burned the full readiness timeout
- register the harness in the two contract tests, which is what surfaced the
SMOKE_BASE_URL bug, and pin the new guards there
- record the second mutation: reverting #7892 moves the longest stall 4-5x
while leaving the long-task total inside the clean range, the exact
opposite of reverting #8750, so both budgets are load-bearing
danielhanchen
added a commit
that referenced
this pull request
Aug 16, 2026
… the stream Two holes left by the measurement window added in the previous commits. A long task carries the start time of its whole task, so appending the user message in the same task that assigned measureFrom stamped runtime startup and the first publish as earlier than the window and dropped them as page load. Hand the append to a later task so the work that begins the stream sorts inside it. longestStallMs was only ever written when a later paint closed the stall, so a freeze that ran to the end of the stream was never recorded: the tail can go missing inside the 90% floor and the quiet-frame loop then calls it settled. Measure the stall in progress while text is still arriving, which is what the number means, and not afterwards, where the settle window's own quiet frames would read as a freeze. Clean runs unchanged (stall 933 to 1,050ms, long tasks 4,749 to 5,159ms over three) and both mutations still caught: #7892 reverted fails the stall at 5,233ms, #8750 reverted fails long tasks at 52,263ms.
danielhanchen
added a commit
that referenced
this pull request
Aug 16, 2026
The stall in progress was measured only while text was still arriving. A freeze that spans the moment the stream ends blocks the frame loop across it, so the first frame afterwards already observes a non-null streamEndedAtMs and the whole frozen interval was skipped. With the lost tail able to hide inside the 90% workload floor, thirty quiet frames then settled the reply and the run reported a short longest stall, which is the one shape this number exists to catch. Cap the interval at the absolute stream-end timestamp instead. A freeze across that moment is recorded in full, and the stall stops growing once there is no more text to wait for, so the settle check's own quiet frames are still not counted as a freeze. The rule moves into smoke-stream-pacing-stall.ts so it can be tested without importing the harness entry, which mounts React on import. The new tests cover the spanning freeze, the settle-window bound, idempotence and late tail paint; restoring the previous rule fails two of the five. Clean runs unchanged (stall 967 to 983ms, long tasks 5,442 to 5,842ms) and both mutations still caught: #7892 reverted fails the stall at 5,017ms, #8750 reverted fails long tasks at 63,687ms.
danielhanchen
added a commit
that referenced
this pull request
Aug 18, 2026
#8969) * Studio: keep the streaming render harness the perf PRs kept rebuilding * Studio: close the false-green paths in the stream pacing harness The long-task total is the metric the budgets turn on, and it reads 0 both when the render is free and when the observer never ran. observe({type: "longtask"}) is specified to abort silently on an engine that lacks the entry type rather than throw, so the try/catch around it never fired: under firefox or webkit the harness scored a perfect zero and exited 0. Detect support with PerformanceObserver.supportedEntryTypes, record it, and fail the run when no long tasks were seen or when throttling was disabled. Also: - add the entry to tsconfig.app.json, which lists the smoke entries one by one, so npm run typecheck actually covers the 270 lines it was reported against - write the JSON under logs/ like every sibling harness instead of dropping an untracked stream-pacing.json in the repo root, and create the directory - treat an exported-but-empty SMOKE_BASE_URL as unset, matching the siblings; it drove "" as the base URL and burned the full readiness timeout - register the harness in the two contract tests, which is what surfaced the SMOKE_BASE_URL bug, and pin the new guards there - record the second mutation: reverting #7892 moves the longest stall 4-5x while leaving the long-task total inside the clean range, the exact opposite of reverting #8750, so both budgets are load-bearing * Studio: budget only the long tasks the stream itself caused buffered: true replays whatever the performance timeline already held, so module evaluation and the first React render landed in the budgeted total: one entry, ~140ms, about 2.6% of a clean run here, and larger on a cold or loaded runner. Nothing filtered by startTime and run() reset nothing, so a slow page load read as a slow renderer. Open the measurement window in run() and drop entries that began before it. Pre-stream share goes 2.6% to 0.00% while the stream's own tasks are unchanged (60 and 52 entries over two clean runs), and reverting #8750 still fails the budget at 52,465ms. * Studio: tighten the stream pacing harness comments Comments and docstrings only, no code change. Every measured number, PR reference and causal reason is kept verbatim. * Studio: check the reply that settled, not the peak it once reached paintedChars is a high-water mark and only ever climbs, so a completion render that truncated the bubble would leave the peak behind and the 90% workload floor would still pass on a DOM that no longer held the reply. Record what is on screen at settlement and check that too. Measured equal to the peak today (24,033 both), so this is a guard rather than a live discrepancy, and it is pinned in the harness contract test. Also count slow frames only inside the measurement window and reset the counter in run(), the same rule long tasks now follow. Contamination measured at 0 of 286 here, but an external server or a slower box need not be 0 and the number is meant to be comparable across them. * Studio: record a stall that never ends, and keep the task that starts the stream Two holes left by the measurement window added in the previous commits. A long task carries the start time of its whole task, so appending the user message in the same task that assigned measureFrom stamped runtime startup and the first publish as earlier than the window and dropped them as page load. Hand the append to a later task so the work that begins the stream sorts inside it. longestStallMs was only ever written when a later paint closed the stall, so a freeze that ran to the end of the stream was never recorded: the tail can go missing inside the 90% floor and the quiet-frame loop then calls it settled. Measure the stall in progress while text is still arriving, which is what the number means, and not afterwards, where the settle window's own quiet frames would read as a freeze. Clean runs unchanged (stall 933 to 1,050ms, long tasks 4,749 to 5,159ms over three) and both mutations still caught: #7892 reverted fails the stall at 5,233ms, #8750 reverted fails long tasks at 52,263ms. * Studio: tighten the comments added since the first pass Comments only, no code change. Every measured number and every causal reason is kept. * Studio: record a freeze that spans the end of the stream The stall in progress was measured only while text was still arriving. A freeze that spans the moment the stream ends blocks the frame loop across it, so the first frame afterwards already observes a non-null streamEndedAtMs and the whole frozen interval was skipped. With the lost tail able to hide inside the 90% workload floor, thirty quiet frames then settled the reply and the run reported a short longest stall, which is the one shape this number exists to catch. Cap the interval at the absolute stream-end timestamp instead. A freeze across that moment is recorded in full, and the stall stops growing once there is no more text to wait for, so the settle check's own quiet frames are still not counted as a freeze. The rule moves into smoke-stream-pacing-stall.ts so it can be tested without importing the harness entry, which mounts React on import. The new tests cover the spanning freeze, the settle-window bound, idempotence and late tail paint; restoring the previous rule fails two of the five. Clean runs unchanged (stall 967 to 983ms, long tasks 5,442 to 5,842ms) and both mutations still caught: #7892 reverted fails the stall at 5,017ms, #8750 reverted fails long tasks at 63,687ms. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make the stream pacing smoke page load again Merging main brought in ASSISTANT_PART_COMPONENTS, which thread.tsx builds at module scope with Text: MarkdownText. Entering the markdown-text -> features/chat -> chat-page -> thread cycle from markdown-text runs that object literal while the MarkdownText binding is still in its temporal dead zone, so the page died with Cannot access MarkdownText before initialization and rendered nothing. Import the chat barrel first, as the app's entry does. The page is also a new HTML entry, so it has to load the crypto polyfill before its module entry like every other one. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothshared@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While an answer is streaming, the chat UI can stop updating for several seconds even though tokens are still arriving from the server. The user sees no progress and then gets a large wall of text all at once.
Long answers with many LaTeX equations trigger this most easily because rendering the Markdown and equations is more expensive. Slower client computers are affected more often because rendering happens in the browser.
This PR keeps the displayed answer updating throughout the stream. In the throttled production test, the median longest freeze fell from 9.86 seconds to 0.34 seconds.
Root cause
Streamdown 2.5 puts normal streaming block updates inside a low-priority React transition. New text keeps arriving as higher-priority work, so React can repeatedly interrupt the block update before it reaches the screen.
The network stream is not stalled. The browser has the new text but continues displaying an older render.
Fix
Use Streamdown's direct-update path for active chat messages through a zero-duration, zero-stagger animation configuration. This bypasses the starved transition without adding a visible animation.
The existing streaming mode, custom code and Mermaid handling, and final completed markup remain unchanged. A regression test protects the direct-update configuration.
Measured improvement
The production Studio chat page replayed the same 13,919-character answer at 1,400 characters per second with 4x browser CPU throttling. The answer contained 59 rendered KaTeX equations and streamed for about ten seconds.
A final run after rebuilding the complete diff produced 49 visible updates and a 0.31-second longest freeze.
Testing
npm run typechecknpm test(372 passed)npm run buildgit diff --checkThis changes only the active chat Markdown renderer. Backend streaming, SSE parsing, and static Markdown previews are unchanged.