[Na0s]

The web-citation fix in #31053 hides the raw markers from #30979, but can also eat the only separator between words: red citeturn0search0blue becomes redblue in display, native Markdown, and copy.

This is a stacked follow-up to #31053, targeting fix/issue-30979, not a second implementation of that fix. It addresses the separator issue already identified in review. Please merge the base first (or fold this correction into it).

Keep the existing delimiter recognizer and use the Markdown AST's next inline source position to decide whether a consumed space/tab is still needed. The source-offset projections insert that separator after closing inline syntax: **red citeturn0search0**blue becomes **red** blue, not the invalid Markdown **red **blue. Block boundaries stop the lookahead. Existing code exclusions and autolink cleanup stay in place; no storage, provider protocol, UI components, or source-link UX changes.

Investigation

• Followed the triage playbook with read-only desktop/provider evidence (macOS desktop/server v0.1.6). The raw markers are in the provider's agent-message text, not a font/CSS problem. Reused [Bug]: Website citations in assistant responses render as garbled text #30979/fix(shared): hide web-search citation markers in assistant messages #31053 instead of opening a duplicate issue; the installed app and live database were not patched.
• Investigated structured metadata, a micromark construct, and the existing shared AST transform. The observed agent-message payload has no source-annotation map. A tokenizer alone also misses a marker swallowed by a GFM bare-URL autolink. fix(chat): render portable Codex source references #22034/fix(web): show Codex source details and retain citation copies #22035 already explore the larger structured-source route, so this PR deliberately stays with a small correction to the existing fix.

Verification

• Added the regression table first. On the base implementation, the initial 18 selected cases produced 12 failures / 6 passes, including plain text, tabs, and both sides of bold spans; the failure was redblue instead of red blue.
• vp test run packages/shared/src/codexMarkdownDirectives.test.ts: 65 passed. Covers all three shared entry points, round-trips projected Markdown through the ordinary parser, and includes directive boundaries, existing/trailing spaces, paragraphs, and headings. Existing code and autolink tests pass too.
• vp run check:changed origin/fix/issue-30979: passed format, lint, both knip phases, typecheck of 34 affected/dependent packages, and 67 tests in 2 files.
• vp test run packages/shared/src/markdownPipeline.test.ts apps/web/src/markdown-incremental.test.tsx: 25 passed.
• No full desktop/mobile app session was manually exercised; verification is of the shared parsing and source-projection behavior, not a native UI/device run. CI still owns the full suites/build.

Created with gpt-6.1-sol in Codex (T3 Code).

[macroscopeapp]

Approvability

Verdict: Approved at df1b364

Macroscope's review found this PR approvable — This preserves word separation when hidden citation markers occur within or between Markdown spans. Its effects are limited to displayed and copied text, with no storage or product-default changes.

^{You can add or adjust custom eligibility rules. Learn more.}

[maria-rcks]

Note

Written by claude-opus-5-5 on behalf of Maria

Takeover update at f7201b053c

the stacked base fix/issue-30979 gained three commits that fix the same separator bug with a different rule, so this pr conflicted with it. merged the base into this branch (merge commit, no rebase) and kept both sides:

• from the base: its delimiter rule (DROPS_PRECEDING_SPACE), link and reference destination cleanup, file citation handling, and **Result**. dropping the space before closing emphasis.
• from this pr: a marker that ends a text node is judged by the next inline node, stopping at the block, and copy/native output puts the kept space after closing syntax (**red** blue, not **red **blue).
• one small addition to make the two agree: the next node is judged by its first content, so red <cite>:codex-followup[blue] keeps its space instead of treating : as punctuation.

before the merge resolution, this pr's separator table failed 8 cases against the base's current code (**red **blue in copy and native output, redblue around follow-up directives).

• blacksmith: vp test run packages/shared/src/codexMarkdownDirectives.test.ts packages/shared/src/markdownPipeline.test.ts apps/web/src/markdown-incremental.test.tsx passed 108 of 108.
• blacksmith: vp run check:changed against the base tip passed format, lint, knip, typecheck of 35 packages, and 83 tests.
• no review-bot findings were open.

unverified at this head: the live web render and copy run below was done at df1b364, before the merge, and was not repeated. native and desktop remain unverified as before.

---

Earlier verification at df1b364 (written by gpt-6.1-sol)

verified as-is at df1b36493c92132edaa70f49a8342fc01bc3c1e9 on linux using the repository’s playwright dependency and /usr/bin/google-chrome.

created a real git repository, added it through the web app, and submitted real codex gpt-6.1-sol turns through the local provider proxy. the prompt asked the provider to emit literal citation-marked markdown, so the boundary cases were deterministic. no replay provider, mocks, seeded messages, or direct database writes. read-only inspection confirmed the assistant response text was identical on main and the pr.

• github/main at b48b8ea2a0: reproduced raw citation markers in rendered and copied output.
• stacked base at 6ab7d10a45: reopening that real response reproduced redblue in plain text, before bold, and after bold. copying also lost the separator.
• pr head: all three cases render red blue. clicking copy message produces the expected markdown exactly, including red **blue** and **red** blue.
• nearby behavior: existing whitespace and paragraph boundaries stay intact; inline and fenced code retain literal markers; both bare and ordinary link destinations remain https://example.com.
• restarted the pr server and reopened the thread: the response persisted and still rendered correctly.
• blacksmith: vp test run packages/shared/src/codexMarkdownDirectives.test.ts packages/shared/src/markdownPipeline.test.ts apps/web/src/markdown-incremental.test.tsx passed all 90 tests in 3 files. pr status reported 30 successful checks, zero failing or pending.

main before:

stacked base before:

pr after, matching response content:

copy and paste interaction, inspected after upload with playback and seeking:

Not verified

native ios/android rendering and copy, and the electron desktop shell were not exercised on this linux web run. native markdown projection is covered by the existing shared tests, but that is not a native-device verification. actual web-search-generated source metadata was not exercised; this run used literal citation markers emitted by the real provider. no source changes or pushes were needed.