Skip to content

iOS: pause ellipses, Apple Intelligence cleanup, and a speech model choice - #887

Open
kitlangton wants to merge 2 commits into
devfrom
ios-polish-ab
Open

kitlangton wants to merge 2 commits into
devfrom
ios-polish-ab

Conversation

@kitlangton

Copy link
Copy Markdown
Collaborator

iPhone dictation: pauses stop splitting sentences, an optional Apple Intelligence punctuation pass, and a choice of speech model, so the three can be compared on a phone.

Before

  • Apple Speech writes ... at a pause. The formatter treated each dot as a sentence end: there's some... Shared idea there.
  • The app always used Apple Speech on iOS 26, even when Parakeet was already downloaded.

After

  • Polished, Casual and Notes treat ... / … as a pause: mid-text it becomes a space and the next word keeps its lowercase start (I, its contractions and acronyms stay as they are). A trailing one is dropped, and Polished adds its period. Verbatim is unchanged.
  • Clean up with Apple Intelligence (Polished only, shown when SystemLanguageModel.default is available, on by default). After the formatter, the app runs one FoundationModels pass that only fixes punctuation and capitalization, joining sentences a pause split.
    • The output is used only if its words (lowercased, punctuation stripped) equal the input's, and only if it answers within 1.5 s. Otherwise the formatter's text stands.
    • The session prewarms when recording starts. Before/after text and latency are logged under cleanup.
    • It runs in the app process, never in the keyboard extension.
  • Speech model: Apple / Parakeet (shown on iOS 26, Apple by default). Picking Parakeet downloads it with the stage bar's progress the first time. If the download fails, the model that was ready stays in use and the row shows the error.
transcribe -> TranscriptFormatter.format(style)        // ellipses -> pauses
           -> if Polished && cleanup on:
                cleaned = model(formatted) within 1.5 s
                use finishPolished(cleaned) if words(cleaned) == words(formatted)

Tests: TranscriptFormatterTests (one real dictated sentence through every style), TranscriptCleanupGuardTests, and a live latency test that skips where Apple Intelligence is missing (the simulator here). FoundationModels is weak-linked, so iOS 18 still launches.

Read: the cleanup and the speech model are A/B settings. Keep or drop them once they've been tried on a phone.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant