Repository navigation
fix(feed): render orphan ghosts only when JSONL stalls past proxy - #66
Merged
Merged
Conversation
Long-form analysis of how ghost works in cc-shell today, all four prior fix attempts and why each regressed, and the case table that drove the layered predicate landing in subsequent commits. Two docs: - 2026-05-07-ghost-system-findings.md — the diagnosis. Inventory of every component file, the dual-owner problem with SemanticStreamingTurn, the stale Phase 3 comment drift, the four fix attempts, and the corrected case table including the tail-sidecar gap an early single-rule predicate missed. - 2026-05-07-ghost-rendering-predicate.md — the superpowers plan for the layered predicate (timestamp gate + sidecar shape filter). Bite-sized tasks; this commit lands the docs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Foundation for the ghost-rendering predicate. Adds
SessionRuntime.lastJsonlEntryAt and stamps it from both ingest
paths:
* useIpcSubscriptions.onSessionJsonlEntries — walks the burst,
parses entry.timestamp ISO strings, takes the max. Includes the
field in the no-change short-circuit so a burst that ONLY
advances the cursor still lands.
* initialHistory bootstrap — same scan applied to the loaded
JSONL tail on resume, so the predicate has the right anchor
against ghost _atp.updatedAt loaded from disk.
history.ts (older-history pagination) deliberately does NOT touch
the field — its prepended entries are by construction older than
runtime.entries.
No visible behavior change yet: nothing reads lastJsonlEntryAt
until selectMergedEntries is rewired in the next commit. See
docs/superpowers/plans/2026-05-07-ghost-system-findings.md for the
full design rationale.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Layered predicate in selectMergedEntries: a ghost is render-
eligible iff ALL of —
1. not superseded
2. orphaned (TTL elapsed without a JSONL match)
3. turnId != currentTurnId (SemanticStreamingTurn owns that)
4. _atp.updatedAt > lastJsonlEntryAt (proxy state past JSONL
tail; structurally distinguishes "JSONL stalled past proxy"
from "JSONL kept writing past this ghost")
5. not sidecar-shaped
Why rules 4 AND 5, not just rule 4: the timestamp predicate alone
fails for the tail-sidecar case (predict-next-prompt fires after
the last real JSONL entry, no later real turn supersedes it via
timestamp comparison). Rule 5 (single ≤200-char text block in an
assistant turn) is the structural backstop. The trade-off — a
real assistant turn that crashed before any JSONL write AND was
a single short text block ("Done.") would also be hidden — is
the same one commit 2a83978 made; this commit keeps the shape
filter while adding the timestamp gate that 686b94e was missing.
Bumps GHOST_ORPHAN_TTL_MS from 3000 to 30000. With the layered
predicate, the orphan flag merely qualifies a ghost as render-
eligible; rules 4 and 5 still gate visibility. So the TTL is now
"how long before we conclude JSONL had its chance," not "how
fast we paint fallback." 30s matches atp's library default and
safely covers slow tool_results without prematurely qualifying
mid-turn.
Test cases land in scripts/test-ghost-fallback.ts (modified
in-place, no new test files): 10 cases covering the predicate
matrix — empty/superseded/unorphaned/current-turn skips, the
timestamp gate (orphan older than JSONL tail = hide), the
sidecar shape backstop (tail orphan with single ≤200-char text
block = hide), substantive text rendering, tool_use rendering
(not sidecar-shaped), fresh-session null-lastJsonlEntryAt path,
and orphanStale TTL semantics.
See docs/superpowers/plans/2026-05-07-ghost-system-findings.md
for the long-form rationale and the case table this predicate
satisfies.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strips the "transcript-first feed with a ghost overlay" / "Phase 3
will delete SemanticStreamingTurn" framings that described an
unfinished plan. Replaces with the actual current behavior: ghost
is a bookkeeping ledger that surfaces in the rendered feed only
when the layered predicate in selectMergedEntries fires (JSONL
stalled past proxy + not sidecar-shaped). The live current turn
is owned by SemanticStreamingTurn directly off the semantic
channel.
ghosts.ts file-level block: rewrites "WHY this exists" to describe
the current two-consumer model (selectMergedEntries fallback +
ghostJournal persistence) instead of the unfinished single-owner
overlay. Adds reference to the findings doc.
TileLeaf.tsx call-site comments: updates the entries={} and
semanticTurn={} prop descriptions to match the layered predicate.
No code change. The full rationale is in
docs/superpowers/plans/2026-05-07-ghost-system-findings.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…t-system.md
Establishes docs/design/ as the canonical home for evergreen
design docs that explain non-obvious subsystems. The README in
that directory states the convention: these files are sources
of truth, NOT changelogs or running notes; do not change them
lightly. Date-stamped plans and forensic write-ups continue to
live in docs/superpowers/plans/.
docs/design/ghost-system.md is the first such doc. It explains:
* what ghost is in plain English (the bridge between the proxy
semantic stream and the durable JSONL transcript)
* why ghost exists (none of the naive alternatives — JSONL-only,
proxy-only, both unconditionally — work)
* the dual-owner relationship with SemanticStreamingTurn that
makes rule 3 of the predicate load-bearing
* the architecture (ASCII diagram + per-file role table)
* the lifecycle (created → updated → superseded / orphaned)
* the five-rule render predicate, reproduced canonically
* provider-aware reconciliation matchers
* reference-stability invariants
* persistence + resume behavior
* the orphan TTL semantics
* Phase 3 explicitly out of scope
* a Warning section listing the four prior fix attempts that
regressed and the specific things future authors must NOT
drop without re-thinking the whole subsystem
Inline comments in every ghost-touching file now point at the
design doc as the canonical explanation:
* src/renderer/src/workspace/ghosts.ts (file-level block)
* src/renderer/src/workspace/mergedEntries.ts (file-level block)
* src/renderer/src/workspace/workspaceState.ts (lastJsonlEntryAt
+ ghosts field docstrings)
* src/renderer/src/workspace/hook/ipc/useIpcSubscriptions.ts
(orphan sweep + ghost mint sites + TTL constant)
* src/renderer/src/workspace/tile-tree/TileLeaf.tsx (call-site)
* src/main/ghostJournal.ts (file-level block)
* src/providers/claude/runtime/claudeSession.ts (sidecar opt-in)
Also tightens the TTL constant comment to reflect that 30000ms
is the right value (long enough that normal slow operations
don't trip it, short enough that real stuck-mid-turn cases
surface within roughly half a minute on resume), and removes
references to a transient "60000ms defensive margin" idea that
was discussed and dropped.
No behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three discrepancies caught in cross-review: 1. Rule 4 of the predicate stated render-eligibility requires `lastJsonlEntryAt !== null && updatedAt > lastJsonlEntryAt`, but the code (mergedEntries.ts) deliberately lets a null lastJsonlEntryAt fall through and the test (test-ghost-fallback.ts case 9) asserts that behavior. The doc now reflects: the rule only gates when lastJsonlEntryAt is known; on a fresh session with no observed JSONL entry, the ghost falls through to rule 5. 2. The Codex reconciliation matcher was described as `(turnId === codexTurnId, blockIndex)`, but ghosts.ts matches by `turnId === codexTurnId` only — a single Codex entry landing supersedes every ghost in that turn (same shape as the Claude message-id contract). The doc now describes the matcher as it is implemented and notes that per-block disambiguation comes from the tool_use_id / call_id fallback. 3. The Live-stuck note read "currentTurn cleared (or hangs)" which was misleading: if currentTurn hangs alive with the same turnId, rule 3 hides the ghost and SemanticStreamingTurn continues to own visibility. The doc now says cleared, and adds a clarifying note that the hang case keeps the live owner in charge. No code change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced Sep 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
selectMergedEntries: a ghost renders iff orphaned + not superseded + not the live current turn +_atp.updatedAt > lastJsonlEntryAt+ not sidecar-shaped. Restores the ghost system's actual purpose (surface the proxy's record when JSONL stalls past it) without recreating the stale-bottom-row regressions from686b94e/2a83978.lastJsonlEntryAttoSessionRuntime, stamped from the live JSONL ingest inuseIpcSubscriptionsand from the resume-time bootstrap ininitialHistory. Compares apples-to-apples against ghost_atp.updatedAt(both wall-clock-when-the-producer-observed-the-event), so the predicate works across crash/restart boundaries.GHOST_ORPHAN_TTL_MSfrom 3 s to 30 s. With the new predicate the orphan flag merely qualifies a ghost as render-eligible — rules 4 (timestamp gate) and 5 (sidecar shape) still gate visibility — so the TTL semantics shift from "how fast we paint fallback" to "how long before we conclude JSONL had its chance."ghosts.tsandTileLeaf.tsx. Replaces them with descriptions that match what ghost actually does today.Why two rules, not just the timestamp gate
Cross-reading the first draft caught a real hole: a single-rule timestamp predicate handles "stale orphan from earlier in the session below newer commits" but fails for the tail-sidecar case (predict-next-prompt fires after the last real JSONL entry, no later real turn supersedes it via timestamp comparison). Production bundles show that's the dominant failure mode. The shape filter from
2a83978is the structural backstop. Trade-off: a real assistant turn that crashed before any JSONL write AND was a single short text block ("Done.") would also be hidden — same trade2a83978made; the alternative is daily orphan title-gen / next-prompt clutter at the bottom of every session.Background
10e4fc5,686b94e,2a83978, the abandoned local v1), the dual-owner problem withSemanticStreamingTurn, and the case table the predicate satisfies.Test plan
npm run test:ghost-fallback(10 cases covering the predicate matrix)npm run buildSemanticStreamingTurnpaints the live response and no orphan ghost shows up at the feed tail within 35 s of completion (covers the predict-next-prompt window).Out of scope
SemanticStreamingTurn, render the live current turn through ghosts + ordered insertion inmergeWithUpstream). Future work; tracked in findings.md §14.isSidecarFlowpredicate for predict-next-prompt — independent improvement, complementary to this renderer-side filter.claudeSession.ts(different problem: bootstrap-replay tool_result spam).🤖 Generated with Claude Code