Repository navigation
fix(core): tighten session title prompt - #57648
Conversation
There was a problem hiding this comment.
No problems found that should block this. I checked the request the title model receives on the base and with this change. The first message is now wrapped in <message> and followed by "Reply with only the title." Regeneration context is wrapped in <conversation> and still fits within 8,000 characters. The space left for recent messages can't go negative, since the original request is capped at 2,000 characters. session-title.test.ts passes (20/20) and packages/core typechecks. I couldn't confirm that real models stop answering, because there are no provider credentials here, so that rests on your benchmark.
Public API unchanged: only the built-in title prompt and the text sent to the model change. Title hooks still receive the request and now see the wrapped text in event.messages.
Problem
Session titles average ~6 words and some aren't titles at all. The title request sends the user's first message as a bare user turn, so on long, instruction-style prompts the small model answers or starts the task instead of naming it:
Here's how I see it. I don't have access to the …We need title only. Need capture investigation empty content …I'll start by exploring the codebase to find both …… for Cloudflare AI gateway</think>Anthropic model IDs …The prompt's examples also teach filler endings (
…investigation,…implementation,…review), and casing drifts between sentence and Title Case across models.Change
PROMPT_TITLEis rewritten in plain prose: aim for three to five words, keep the detail that distinguishes the session (version, provider, comparison target) and drop generic words to make room, sentence case with identifiers kept verbatim, and the input is something to name, not answer. Examples now model short titles.Before / after, same first message and model:
Benchmark
First prompts from 30 real sessions (8 short, 12 medium, 10 long), replayed on six OpenCode-provider models at their lowest reasoning variant (
claude-haiku-5-5#none,gpt-5-nano#minimal,gpt-6-luna#none,deepseek-v4.1-flash#low,minimax-m3,gemini-3.5-flash-lite#minimal), 2 runs each = 360 titles per prompt.claude-opus-5-5,gpt-6.1-sol) scored the new titles level with the current ones (average rank within 0.1, judges split on order). A tighter 3–4 word target was shorter still but ranked clearly worse because it dropped the distinguishing detail.Out of scope
/renamestill publish the samesession.renamedevent, so the two can't be told apart afterward.gpt-5-nanostays the wordiest model (5.6 words) and sometimes starts lowercase.Verification
bun test test/session-runner.test.ts test/session-title.test.ts test/session-generate.test.tsinpackages/core: 255 passbun testinpackages/core: 6684 pass, 8 fail, all ShellTool bash process-substitution cases that fail on this macOS machine and pass on Linux CI; unrelated to this changebun typecheckinpackages/core: pass