Skip to content

Integration batch E (#1349, #1353, #1364, #1383) - #1404

Merged
Juliusolsson05 merged 43 commits into
mainfrom
integration/batch-2026-09-27-e
Sep 27, 2026
Merged

Juliusolsson05 merged 43 commits into
mainfrom
integration/batch-2026-09-27-e

Conversation

@Juliusolsson05

@Juliusolsson05 Juliusolsson05 commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

Integration batch E: owner-approved merge mode. Members were merged --no-ff onto 95aff39 (built from a stale local main ref; q103), then current main 5e22c7b (batch D) was merged in at 870c607. D and E touch no common files. The earlier green CI on b986aa0 is void, and the gate is exact-head CI on 870c607.

Every member passed merge-gate --member: 3 reviews or a manager-verify at the cap, a disposition, labels, and green exact-head checks. #1349 and #1353 carry manager verifications and stated residuals.
Merge only after green CI on this exact head.

🤖 Generated with Claude Code

Juliusolsson05 and others added 30 commits September 26, 2026 19:23
…saves, daily profile (#767)

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The background refresh (at least every 60 s) re-stamped updatedAt,
stringified the whole index (30 MB+ for heavy users) and replaced the
file even when every candidate was a cache hit, and the new timestamp
invalidated the summary cache that exists to spare the 10 s UI poll.
A refresh now persists only if something was parsed, a parse failed or
the entry count moved; otherwise it records only when it checked. The
store persists the caller's updatedAt instead of stamping its own.
First tests for WorktreeActivityIndex (fail-first on origin/main).

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…are on; document the flags

The feed-debug ring was shipped to disk once a second for every session
in every build (up to 128 MiB per session file) with no switch. It is
now persisted only with AGENT_CODE_DEV_DEBUG=1 or the Persistent
Aggressive Debug Logs setting. The ring itself still records, and Save
Debug Logs reads the ring, so bug reports are unchanged; only the
after-crash copy needs a switch. Turning a switch on persists the
retained tail on the next tick.

The README gains a Diagnostics flags section: each flag and setting,
its default and cost, and the recommended daily profile (all off,
DEV_DEBUG only on demand). Proxy streaming is documented as the live
transport, not a diagnostic.

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ture (#1296)

Refs #1296

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…corpus

The fixture kept every field verbatim apart from the screen mirrors,
which committed the home path, private project names, queued prompts,
drafts, prompt suggestions, subagent task descriptions and streaming
baselines. The row model reads none of that text. Text fields are now
filled with x at their own length; the home user and non-Agent-Code
project names are same-length placeholders (keys included). The
extractor applies the same pass, so a regeneration stays redacted. The
original content is still in main's history (owner decision).

Refs #1296

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y the recording's

The resolver scans /opt/homebrew/bin and /usr/local/bin whatever HOME
and PATH say, so on a Mac with a Homebrew claude/codex/opencode the live
probe finds a tool the clean recording did not, and the comparison
failed for a reason unrelated to the code. A row either side found at a
machine-wide path outside the simulated HOME and app root is a fact
about that machine and is not compared.

Refs #1296

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… switching persistence off

Review of #1349:
- "Nothing parsed and the same count" missed a swap (B leaves, C comes
  and goes, B returns unchanged from the LRU), so the stale set stayed on
  disk. The index now tracks its on-disk key set and persists when the
  set of paths differs.
- Turning feed-debug persistence off ran the effect cleanup's final
  flush, writing one more batch; the cleanup now flushes only while the
  switch is still on (a real unmount).
- Tests pin the in-memory generation through the served summaries and
  the provider-session cache guard (both mutations survived before).
- README: AGENT_CODE_PERF adds spans and traces; the Performance
  Monitor's baseline runs regardless. The command-palette text for the
  setting names the feed-debug log too.

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…re's no-restamp

Review b of #1349: with the key-set comparison, a previously indexed
transcript that fails to parse already counts as a change (it leaves
the set); counting skipped files forced a full index rewrite on every
refresh while an unindexed file stayed unreadable. A store-level test
now pins that saveWorktreeActivityIndex keeps the caller's updatedAt
(the old test only caught a restamp when two Date.now() calls
straddled a millisecond).

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tart, subagent watcher)

- githubCli.system: drop the "timeout + 2 s" elapsed assertion around a
  real gh spawn; the production timeout and the test's own limit already
  catch a hang, and a loaded machine could exceed it legitimately.
- BrowserPocketHost: wait for the observed restart (nextCrashDelay =
  250 ms) instead of a fixed 400 ms sleep.
- SubAgentWatcher: await refresh(), which resolves after a pass that
  started after the call, instead of polling a 1.5 s deadline.

Refs #1296

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Electron harnesses use a plaintext secrets codec (a journey must
never touch the real keychain) in a fixed os.tmpdir()/agent-code-
harness-secrets-<pid> directory that was never removed. Each run now
gets a mkdtemp directory, removed on exit (covering the app.exit(1)
failure paths) and before app.quit().

Refs #1296

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #676

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t the rest

The decision log was append-only for a session's lifetime and every
departure copied the whole array, so a long-lived session paid memory
and O(n) per departure for history nothing reads (no UI, no debug
bundle). One helper now appends, keeps the newest 200 decisions and
counts the evicted ones; conservation is restated exactly as
decisions.length + droppedDecisions === nextSeq. Fail-first: 1,000
queue/remove cycles kept 1,000 decisions on origin/main.

Refs #676

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 2 of the #1349 review: the LRU only aged entries out, so for light
users (whose summaries are built from the LRU) a deleted transcript kept
appearing in the Worktrees summary, and one could return as a cache hit
without being re-parsed. Every save now trims the LRU to the on-disk key
set, which also makes any key new to the index a cache miss; the
explicit key-set comparison is therefore unreachable and is removed.

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he index

Review c of #1349 (round 2): re-adding the skipped-file trigger passed
every test. A discovered transcript that keeps failing to parse is not
index content; the new case keeps the file bytes and mtime unchanged
across a refresh with one skipped candidate.

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s (review of #1353)

- first-run: a live machine-wide row is skipped only when its executable
  is the provider's own command; a resolver returning /opt/homebrew/bin/grok
  for opencode now fails instead of being excused.
- githubCli: a PATH-first gh shim makes the real runner's answer knowable;
  a runner that never spawns gh now fails (the real-gh case accepts null).
- activityRow: a runtime that needs the user as claude must not as a
  terminal; the title wins the name when a Goal also exists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…daction policy

Review of #1353 (q70): the committed runtime fixture was not the
extractor's output (hand-edited placeholders), the placeholder merged
same-length projects, and branch names plus worktree suffixes of private
repos survived. Also `startup` in live-fleet.

- scripts/agent-activity-redaction-policy.ts: one distinct same-length
  placeholder per project, rest of a private path x-filled, branch filled.
- extractor --redact-from: re-run only the privacy pass; the committed
  file is byte-identical to its output over 15e43ab^.
- testing/unit/agentActivityRedactionPolicy.test.ts pins the policy and
  that the committed file is a fixed point of it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the exact public repo (q74)

- --redact-from refuses without an explicit, plain --home-user, checked
  before anything is read or written; both modes refuse output that still
  names a home directory other than the placeholder (count only, never
  the name). A missing flag used to fall back to the user RUNNING the
  script, leaving the recorder's path in the tracked fixture.
- The public-name exemption is the exact `agent-code` segment, not a
  prefix: `agent-code-private` (and sibling agent-code-<task> checkouts)
  are redacted.
- Regenerating exposed a data-loss bug: the x-filled tail made
  same-length paths under one project one object key. Each tail segment
  now gets its own placeholder, and a key collision refuses the file.
- Fail-first: the new unit and process-level system tests fail on
  2f93023's scripts. The fixture is regenerated (structure identical,
  every length preserved, no private names).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
assertNoForeignHome accepted any home segment the placeholder starts
with: for an 8-character recorder the placeholder is `fixture-`, so an
unredacted `/Users/fixture/...` passed and a wrong --home-user could write
it. Slash paths must now BE the placeholder; for Claude's dash-encoded
dirs every `-Users-` must be followed by the whole placeholder and a dash
(a user name can contain dashes, so ambiguity refuses).

Fail-first: the unit counterexample and a process-level wrong-user case
both fail on 42ce78e's guard. Also pins text-field rules inside arrays;
key redaction is killed by the same-length-keys test (reviewer c, round 2).
Regeneration unchanged (byte-identical).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pelling (q76/q78)

Round-3 privacy review of #1353 (a) and steering q78:
- a foreign user spelled like the placeholder (`/Users/fixture-`) passed
  the output guard; a new source check before redaction requires every
  home to be the named recorder;
- the dash encoding has no delimiter after the user, so
  `-Users-recorder-secret-...` passed as `recorder`; a dash home is now
  accepted only when a standard home folder follows the recorder AND the
  source names that folder canonically, else it refuses as ambiguous;
- lowercase, backslash, JSON-escaped, %2F and ~name home spellings pass
  through the pass untouched, so the output guard refuses them;
- a bare --redact-from (or an unknown flag) fell through to the LIVE
  extraction; it now exits before any read or write.

Fail-first: the four new process cases fail on 913f5e5. Regeneration
unchanged (byte-identical).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s stay refused (q78)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… final round)

Every privacy round found another input shape a pattern-based pass lets
through (latest, round-3 c: a private project outside Development/). This
mode exists to reproduce the audited committed file, so it now accepts
only the recorded pre-redaction corpus (15e43ab^, blob d265340, by
sha256), after the source home check and before any write. Refusals exit
2 with the curated message only.

A process test reproduces the committed fixture byte for byte from the
git blob (skipped on a shallow checkout). The live extraction's
remaining limits are stated in the plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Final privacy round a (a valid blocker, steering q79): a no-argument live
run wrote `/Users/fixture-home/Projects/secretproject/private-task` into
the tracked runtime-states.json, because the pattern-based pass only
rewrites projects under Development/.

- Live extraction requires --out <path>, refuses the tracked fixture and
  any path inside the repository that git would commit
  (temp/fixture-staging/ is ignored), validated before any bundle is read.
- The tracked fixture changes only through --redact-from (the pinned
  corpus) or a manual copy after a key-by-key audit; the header says so.
- Real process tests from the A reproduction: a no-argument run is
  refused and the tracked file is unchanged (red on 6b042b7); each live
  guard (argument, tracked path, git-ignored, source homes, output homes)
  turns a test red when removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y (q80)

Steering q80 (a scope narrowing, not another hardening round): an --out
staging path validated with git check-ignore was bypassed by a symlink at
or above it, which overwrote the tracked fixture. Live mode now takes no
arguments and writes only into a directory it just created with
mkdtemp(os.tmpdir(), "agent-activity-staging-") (flag wx), prints the
path and exits. liveOutputPath and the git logic are deleted.

The tracked fixture changes only through --redact-from (the pinned
corpus) or a manual copy after a key-by-key audit. Process tests: a live
run with a symlink planted at the old staging path leaves the tracked
file unchanged and stages into the fresh directory; --out is refused;
foreign-home bundles stage nothing. Mutations killed: staging at the old
fixed path, arguments not refused, the source check removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…q81)

os.tmpdir() follows $TMPDIR, so a TMPDIR pointing into a repository (or a
symlink into one) staged the unaudited live extraction inside the working
tree, where `git add -A` would commit it. Before any bundle is read, the
temp root is realpath'd and refused when `git rev-parse --show-toplevel`
succeeds there (or when git cannot run).

Process test: a private bundle with TMPDIR symlinked into a git repo
refuses and leaves no private bytes in the repo. Red on 0d9a02a and with
the guard removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (final check)

Final check of #1353 (a and c): the q81 guard asked `git rev-parse` and
read every non-zero exit as "outside a worktree", but git's discovery
can be switched off from the environment (GIT_CEILING_DIRECTORIES above
a subdirectory, a bogus GIT_DIR), and the unaudited file was staged
inside the repo. The temp root is now realpath'd and refused when it or
any ancestor holds a `.git` entry; no git process, nothing an env var
can change.

Process tests with both environments are red at 980a78b and with the
walk removed. Residuals stated in the plan: a concurrent same-user race
on the fresh directory (needs openat), and `wx` being defensive only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…by descriptor (q82)

Round 5 a (a valid blocker per the manager, q82): a concurrent process
could swap the fresh staging directory for a symlink between mkdtemp
and the by-path write, redirecting unaudited private bytes into a git
worktree. There is no staging directory now: writeStagedFile opens one
new agent-activity-staging-<uuid>.json under the checked temp root with
wx and 0600, writes through that handle and closes it; renaming or
replacing the path afterwards cannot move the bytes.

A unit test swaps the path for a symlink between the open and the write:
the victim stays untouched and the bytes are in the created file.
Mutations killed: write by path, mode 0644.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l invariant (review)

Round 1 of #1364 (a, b, c):
- `decisions.length + droppedDecisions === nextSeq` was false for valid
  states (pending items, stale marks). droppedDecisions now counts
  evicted decision RECORDS; the equality is asserted only for drained,
  never-stale sessions, and says why.
- A recent-only window evicted every decision around rows still stranded
  in the queue. While any item is pending the log may grow to
  QUEUE_DECISION_CEILING (2000); with the queue empty it trims to the
  window, raised to 500 (the corpus already has a 175-decision session).
- Boundary tests for a single large batch and the popAll path.
- markStaleWhenIdle's "anything departed" guard counts evicted records.

Mutations killed: batch counted as one, popAll bypassing the bound, no
pending ceiling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ling (review round 2)

Round 2 of #1364 (c): the 2000 ceiling only postponed the loss; after
2,200 unrelated departures none of the 164 decisions around two stranded
rows remained. An episode opens when an item enters an empty queue and
closes when it is empty again; while open, its first decisions move into
episodeHead (up to 200) instead of being evicted. Head and log share the
ceiling; the head is released and counted when the episode closes.

Also pins the idle-marking path (stale marks bounded, and by the pending
queue). Mutations killed: no episode head, stale marks bounded by the
empty window, stale marks unbounded, head outside the ceiling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Juliusolsson05 and others added 12 commits September 27, 2026 00:22
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Codex ghost is keyed by the proxy response id (resp_...), while the
committed rollout entry only carried the rollout turn UUID (codexTurnId).
reconcileUpstream compared the two, which can never match. Text blocks
have no tool id to fall back on, so every Codex text ghost orphaned
after 30 s and stayed hidden only by render rules 3 and 4.

Both sides do carry the provider item id (msg_/rs_/fc_/ctc_): the live
block has it from the proxy adapter, and the rollout has it on every
response_item. On six real 0.157.1 sessions, all 45 msg_ ids seen on the
proxy were present verbatim in their rollouts. The mapper now stamps
codexItemId, the ghost records its block's itemId in its context, and
reconcileUpstream matches on it. The dead turn-id branch and the stale
comments claiming the mapper stamped the response id are gone.

Fail-first: a real recorded message (proxy SSE frames plus the rollout
record) driven through the real adapter, fold, mapper and reconciler.
Red on main; each half of the fix is pinned by mutation.

Fixes #1231

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… match

Review a of #1383: rollout.ts, ghosts.ts and useIpcSubscriptions.ts
still said Codex ghosts reconcile by (turnId, blockIndex) or by turn id.
They reconcile by provider item id now, and codexTurnId's remaining
consumer is renderUnits.ts.

Refs #1231

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ot matched

Review c of #1383: the header listed rs_ ids as matched, but reasoning
items map to no feed entry, so reasoning ghosts still orphan as before.
The identity test now maps through createCodexTranscriptEntryMapper, the
wrapper all three production ingest surfaces use.

Refs #1231

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Juliusolsson05 Juliusolsson05 added the type:chore Maintenance, deps, tests, docs label Sep 27, 2026
@Juliusolsson05
Juliusolsson05 marked this pull request as ready for review September 27, 2026 11:49
@Juliusolsson05
Juliusolsson05 merged commit 505efa6 into main Sep 27, 2026
2 checks passed
@Juliusolsson05
Juliusolsson05 deleted the integration/batch-2026-09-27-e branch September 27, 2026 11:49
@Juliusolsson05

Copy link
Copy Markdown
Owner Author

Batch disposition (B6, posted after merge; q105)

Merged as 505efa6 from exact head 870c607.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:chore Maintenance, deps, tests, docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant