Skip to content

perf(provenance): compute transcript provenance incrementally (#1475) - #1478

Merged
currentbits merged 1 commit into
mainfrom
coder/perf-f-provenance-incremental-8ba51f
Oct 5, 2026
Merged

currentbits merged 1 commit into
mainfrom
coder/perf-f-provenance-incremental-8ba51f

Conversation

@currentbits

Copy link
Copy Markdown
Owner

Part of #1475 (renderer finding 1 in docs/perf/2026-10-05-renderer.md). provenanceById was recomputed over the whole transcript on every streaming push: about 30% of renderer JS while streaming into a big open thread, and ~48 ms per big-thread switch.

Change

  • New provenanceByMessageId(messages) in src/provenance.ts, used by useTranscriptAnnotations:
    • One pass carries each turn's tool refs forward, instead of rescanning the turn for every assistant message.
    • Each tool call is classified, and its input JSON-parsed, once per object.
    • An assistant message whose object and accumulated refs are unchanged returns its cached result.
  • The renderer keeps unchanged message objects identical between pushes (the patch replaces only the changed tail), so the cache hits during streaming.

Checked independently on real data

  • A read-only benchmark over the three largest real transcripts (1,019–1,062 messages, mostly tool rows) ran 200 simulated streaming pushes each, appending to a new last assistant message immutably, as the store does.
  • Equivalence: at every push, the new map deep-equals the old per-message computation, with 0 mismatches over 600 pushes.
Transcript Old per push (median) New per push (median) Speedup
1,062 messages 1.25 ms 0.03 ms 36.6×
1,019 messages 1.38 ms 0.04 ms 34.6×
1,039 messages 0.11 ms 0.03 ms 3.7×

These are plain-Node timings, without React rendering or the profiler; PM2's in-app figure was larger. The ratio is what carries over.

Tests: test/provenance.test.ts covers streaming, rewind and edit-and-resubmit equivalence. It fails on main, where the function doesn't exist. Renderer suite 2705/2705, typecheck clean.

🤖 Generated with Claude Code

One pass carries each turn's tool refs forward instead of rescanning
the turn per assistant message, each tool call is classified (and its
input JSON.parsed) once per object, and an assistant message whose
object and accumulated refs are unchanged returns its cached result.
Output deep-equals the per-message definition; tests cover streaming,
rewind and edit-and-resubmit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@currentbits
currentbits merged commit 191ad88 into main Oct 5, 2026
6 checks passed
@currentbits
currentbits deleted the coder/perf-f-provenance-incremental-8ba51f branch October 5, 2026 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant