Skip to content

feat: Meeting Domain prompt support for Whisper (vocabulary hints via initial_prompt)#475

Open
Polsemaker wants to merge 3 commits into
Zackriya-Solutions:devtestfrom
Polsemaker:pr/meeting-domain-prompt
Open

feat: Meeting Domain prompt support for Whisper (vocabulary hints via initial_prompt)#475
Polsemaker wants to merge 3 commits into
Zackriya-Solutions:devtestfrom
Polsemaker:pr/meeting-domain-prompt

Conversation

@Polsemaker
Copy link
Copy Markdown

@Polsemaker Polsemaker commented May 20, 2026

Description

Adds a "Meeting Domain" feature that lets users provide vocabulary hints (custom names, jargon, technical terms) to bias Whisper transcription via its initial_prompt parameter.

Domains are plain-text files in ~/Library/Application Support/com.meetily.ai/domains/ (falls back to ~/.meetily/domains/). When a domain is active, its contents are forwarded as initial_prompt to whisper-rs at every transcription call site. No domain selected → unchanged behavior.

UX surfaces:

  • New "Meeting Domain" section in Transcript Settings with a dropdown + a Manage… dialog (list / edit / delete / open folder / char counter)
  • A "Domain" button in the transcript header next to the existing Language button, so users can switch mid-recording
  • A "Meeting Domain" dropdown in the Retranscribe dialog with a "None" option, so users can A/B different hint files without changing their global setting
  • All Domain UI is hidden when Parakeet is the active provider (Parakeet doesn't support initial_prompt)

Implementation notes:

  • New module: frontend/src-tauri/src/meeting_domain/mod.rs (file CRUD, name validation, ~1000-char / ~224-token truncation matching whisper.cpp's hard limit)
  • Global MEETING_DOMAIN state mirrors the existing LANGUAGE_PREFERENCE pattern; Tauri commands surface get/set/list/save/delete to the frontend
  • initial_prompt is threaded through transcribe_audio / transcribe_audio_with_confidence so every whisper invocation receives the same hint (live worker, standalone command, parallel processor, audio import, retranscription)
  • ConfigContext persists selectedDomain to localStorage and syncs to Rust on mount, mirroring selectedLanguage

Related Issue

Fixes #474

Type of Change

  • New feature
  • Bug fix
  • Documentation update
  • Performance improvement
  • Code refactoring
  • Other

Testing

  • Unit tests added (7 tests in meeting_domain::tests covering name validation, listing, save/delete round-trip, and truncation)
  • Manual testing performed on macOS with NB-Whisper-large (Norwegian) — confirmed proper nouns and domain jargon are recognized correctly when the matching domain file is active, and that unchecking the domain restores baseline behavior
  • cargo test meeting_domain — 7/7 pass
  • cargo build — clean (warnings are pre-existing in unrelated files)
  • tsc --noEmit — 0 errors

Documentation

  • Documentation updated
  • No documentation needed for initial review — happy to add a docs page once the design is accepted

Checklist

  • Code follows project style
  • Self-reviewed the code
  • Added comments for complex code
  • Updated README if needed
  • Branch is up to date with devtest
  • No merge conflicts

Screenshots

Settings → Transcription → Meeting Domain (with Manage dialog)

Meeting Domain section in Transcription settings

Transcript header — Domain button next to Language

Domain button in transcript header during recording

Retranscribe dialog — per-retranscription Meeting Domain override

Retranscribe dialog with Meeting Domain dropdown

Additional Notes

  • ~224 token (~1000 char) cap on initial_prompt is enforced with a graceful truncation + log warning, matching whisper.cpp's hard limit.
  • Domain UI is provider-aware: hidden for Parakeet to avoid suggesting a no-op control.
  • Selected domain is read fresh at each transcription start, so editing a domain file mid-session takes effect on the next recording without a restart.
  • Design intentionally orthogonal to [FEATURE] VibeVoice-ASR for speaker diarization and domain-specific terminology #335 (VibeVoice-ASR) — this is a small biasing knob on the existing Whisper path, not a new engine.

Petter Schultz and others added 3 commits May 20, 2026 10:22
Adds a new meeting_domain module that loads vocabulary hint files from
~/Library/Application Support/com.meetily.ai/domains/ (or ~/.meetily/domains/
as fallback) and forwards them as initial_prompt to whisper-rs. This biases
the decoder toward custom vocabulary (proper nouns, client names, jargon)
without changing default behavior when no domain is selected.

- New meeting_domain module with file CRUD, name validation, and ~1000-char
  truncation (whisper.cpp's ~224 token cap)
- Global MEETING_DOMAIN state + Tauri commands mirroring LANGUAGE_PREFERENCE
- initial_prompt threaded through transcribe_audio /
  transcribe_audio_with_confidence at all whisper call sites (live worker,
  standalone command, parallel processor, audio import, retranscription)
- Unit tests for validation, listing, save/delete, and truncation

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a Meeting Domain dropdown to the Transcript Settings tab and a "Domain"
button next to the existing Language button in the transcript header so users
can switch vocabulary hints during an active recording.

- MeetingDomainSettings: dropdown + Manage… dialog (list, edit, delete,
  open folder), char counter, refresh button
- ConfigContext: selectedDomain state with localStorage persistence and
  mount-time sync to Rust (mirrors selectedLanguage pattern). Switched
  TranscriptModelProps/SelectedDevices to type-only imports so the heavy
  TranscriptSettings module isn't pulled into the layout chunk
- Wires meetingDomainSettings modal into useModalState / SettingsModals
- Renders Domain button only when provider is localWhisper (Parakeet
  doesn't support initial_prompt)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Enhance dialog now exposes a Meeting Domain dropdown alongside Language
and Model. It defaults to the currently selected global domain but can be
overridden (including to "None") for a single retranscription, so users can
test different vocabulary hints without changing their global setting.

- start_retranscription_command accepts an explicit meeting_domain parameter
  that bypasses the global current_meeting_domain_prompt() lookup
- Dialog only shows the dropdown for Whisper (Parakeet ignores initial_prompt)
- Domain list is fetched once when the dialog opens

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant