Repository navigation
Description
Problem
Agent Code wraps speech-to-text input in <stt ...> because dictation can contain transcription mistakes. That warning is useful, but it is reactive: the model sees a possibly-wrong transcript after Deepgram has already guessed.
Deepgram supports vocabulary/context biasing (keyterm for current Nova-3/Flux flows, and keywords with boosts for older model flows). Agent Code should expose a safe feedback loop where agents can inspect and improve the dictation context used for future speech-to-text.
Example failures this should address:
MCPbeing heard as unrelated wordspull request/PRbeing mangled- project names, branch names, issue numbers, provider names, and repo-local terms being misheard
- casing/formatting knowledge such as
Codex,Claude,Deepgram,OpenAI,GitHub,TypeScript
Proposed shape
Add an optional built-in MCP domain, tentatively dictation_context, that lets an active agent read and update a bounded dictation vocabulary/context store.
The MCP server should expose tools like:
dictation_context_read- returns currently active keyterms/keywords, formatting rules, model/provider mode, and limits
dictation_context_add_terms- adds short technical/domain terms with optional aliases and rationale
dictation_context_remove_terms- removes stale or bad terms
dictation_context_record_correction- records that a misheard phrase should map to a preferred term in future context
- optionally
dictation_context_suggest_from_workspace- proposes terms from repo names, branches, package names, open issues, and common Agent Code concepts
Deepgram integration
Current code path:
- streaming Deepgram URL is built in
packages/agent-voice-dictation/src/speech/deepgram/streaming.ts - Agent Code starts streaming in
src/main/ipc/dictation.ts - current streaming URL only sets
model
Desired behavior:
- For Flux / Nova-3-compatible flows, append repeated
keyterm=<term>query params. - For older Deepgram model flows, use
keywords=<term>:<boost>only where that API is appropriate. - Batch fallback should receive equivalent context, not only the streaming preview.
- Dictation debug logs should record which context version/hash was used, but must not leak API keys or full private audio.
Storage / safety constraints
- Store context under Agent Code state, not inside project files by default.
- Keep a small, bounded active list, for example top 100 terms.
- Terms should be short words or noun phrases, not arbitrary prompt text.
- Normalize duplicates and casing.
- Track provenance: user-added, agent-added, inferred, timestamp, optional rationale.
- Agents should be allowed to improve vocabulary, not inject broad behavioral instructions into dictation.
- Provide a way to inspect and clean up stale terms.
Acceptance criteria
- A built-in MCP domain exists for dictation context and can be enabled for selected agents.
- Agents can read the active dictation vocabulary/context through MCP.
- Agents can add/remove bounded terms through MCP with validation.
- Deepgram streaming and batch transcription receive the active context.
- Debug traces show the context version/hash and term count used for each dictation session.
- Existing dictation works unchanged when no context terms are configured.
Related code
packages/agent-voice-dictation/src/speech/deepgram/streaming.tspackages/agent-voice-dictation/src/speech/deepgram/index.tssrc/main/ipc/dictation.tssrc/mcp/runtime/createBuiltInMcpServer.tssrc/mcp/shared/types.ts
Activity
Juliusolsson05 commentedon Sep 27, 2026
Marked
later(owner decision, 2026-09-27). A good idea, but not worth the work right now. The triage's suggested first step, if it's picked up: a user-edited keyterm list passed to both Deepgram streaming and batch. Letting agents edit it over MCP is a separate, later decision.