fix(thread): a trim's scrollTop clamp no longer unsticks the transcript (#1475)

docs/perf/2026-10-06-renderer-heap.md

@@ -0,0 +1,45 @@
+# Renderer heap growth during a streaming soak (#1475)
+
+Follow-up to `2026-10-06-end-to-end.md` §5: after #1482 the open transcript's DOM stays flat (~13k nodes) across a six-stream soak, but the JS heap still went 40.8 → 52.7 MB.
+
+## Method
+
+- **Bundle:** a development-mode build with production React: `NODE_ENV=production vite build --mode development` with `import.meta.env.DEV` forced on, so devCoder stays in. It ran in headless Chromium (playwright 1.63). No Electron was launched and the live data dir wasn't touched.
+- **Seed:** the real 3.3 MB, 1,055-message thread `fae77a0b`, read from PM3's frozen snapshot and injected into devCoder.
+- **Streams:** each answer is ~38 KB of PM2's markdown parts. It's pushed as real `thread:updated` patches (`messagesFrom` tails) through the renderer's own listener, 80 pushes per answer.
+- **Sampling:** `HeapProfiler.collectGarbage` ×3, then `Runtime.getHeapUsage`, `Memory.getDOMCounters`, and heap snapshots at streams 0/6/12 (also 0/3/6). Snapshots were diffed by constructor, by stable node id, and by "retained only through X" reachability.
+
+## Retainers
+
+| Retainer | Growth | Bounded? | Action |
+|---|---|---|---|
+| `IntersectionObserver` (auto-animate `observePosition`, created from its debounced `updatePos` timer after the row left the DOM), root = `document.documentElement`, kept by Blink's `IntersectionObserverController`. It pins the detached sidebar card / shelf / shelf toggle and its listeners. | +1 observer, +38 DOM nodes, +5 listeners per stream; 4–5 per status flip in the dev server | **No.** It lives for the whole session, one per status change. | **Fixed** (`skipDetachedObserve`): every leaked observer was created on an already-detached target (14/14), so `observe()` skips detached targets. |
+| V8 compiled code (`InstructionStream`, `TrustedByteArray`, `Code`) | ~1.1 MB s3→s6, ~1.6 MB s6→s12, in steps (tier-up) | Settles | None |
+| Streamed answer strings in `detail.messages` | ~38 KB per answer | Is the transcript | None |
+| `threadDetailCache` | 0 during the soak (it replaces the open entry) | Yes: 8 entries / 8 M chars | Stays |
+| `provenance.ts` `toolRefCache` / `messageCache` | 0 | WeakMaps keyed by message objects | Stays |
+| `MarkdownChunk` memo / `markdownParses` | 0 (a counter only) | Freed with unmount | Stays |
+| PathLinks `cacheRef` | 0 here (the soak repeats the same paths) | Per distinct path string; small | Stays |
+| Fiber alternates, `patchSeqRef` | 0 | Yes | Stays |
+| `PerformanceMeasure` (Blink user-timing buffer) | ~3 MB per stream | **No, but React development builds only** (React 19.2 performance tracks); the production bundle has no `performance.measure` | None: it affects `npm run dev` sessions only |
+
+## Numbers (12 streams, real thread, production React)
+
+| After stream | 0 | 3 | 6 | 9 | 12 |
+|---|---|---|---|---|---|
+| BEFORE heap MB | 26.96 | 28.40 | 29.78 | 31.59 | 31.87 |
+| AFTER heap MB | 22.68 | 24.09 | 25.44 | 27.23 | 27.48 |
+| BEFORE DOM nodes / listeners | 6,669 / 294 | 10,569 / 1,104 | 10,683 / 1,119 | 10,797 / 1,134 | 10,911 / 1,149 |
+| AFTER DOM nodes / listeners | 6,669 / 294 | 10,455 / 1,089 | 10,455 / 1,089 | 10,455 / 1,089 | 10,455 / 1,089 |
+| live `IntersectionObserver`s, BEFORE → AFTER (s0 / s12) | 12 / 24 → 12 / 12 | | | | |
+
+- **Stream-0 gap:** the 4.3 MB lower AFTER heap at stream 0 is a harness artifact: devCoder's backend map held an extra copy of the seed in that run. It is not credited to the fix.
+- **Heap after stream 8:** it grows +0.08–0.15 MB per stream, which is the answer text, counted twice (devCoder's backend copy plus the renderer's).
+- **DOM:** stream 0 → 3 is the tail window filling to `TRANSCRIPT_CHAR_BUDGET`, then it is flat.
+
+## Not reproduced / risks
+
+- **Electron's +12 MB:** the end-to-end run's +12 MB at streams 2–3 doesn't reproduce in the browser, where the same window fill costs ~1.5 MB. The paths I couldn't exercise without launching Electron are the preload bridge, real runner patches and a 2,000-row thread list. The leaked observers retain only ~0.3 MB per 12 streams here; with the real sidebar each detached row is bigger.
+- **Renderer RSS** (295 → 416 MB) is Chromium allocator, layout and GPU memory, not JS heap. PM2's `vmmap` already showed a malloc zone of only 21 MB.
+- **Intermittent pin loss:** in 2 of ~10 dev-server runs, the transcript window never trimmed: nodes climbed ~2k per stream, exactly the pre-#1482 shape. The likely cause is the stick-to-bottom pin being lost (`useStickToBottom` / `ThreadView` window start). It is not addressed here.
+- **Prototype patch:** the fix patches `IntersectionObserver.prototype.observe` app-wide. auto-animate is the only user. A future user that observes an element before inserting it would silently get no callbacks.

docs/perf/2026-10-06-window-trim.md

@@ -0,0 +1,44 @@
+# Transcript window stops trimming mid-soak (#1475)
+
+Follow-up to `2026-10-06-renderer-heap.md` ("Intermittent pin loss"): in 2 of ~10 dev-server soaks the open transcript's tail window never trimmed again, and DOM climbed ~2k nodes per streamed answer, the pre-#1482 shape.
+
+It is an app bug, not a harness artifact.
+
+## Reproduction
+
+Vite dev server plus headless Chromium (npx-cached playwright), with `main` + #1489 + #1490. The real big thread (fae77a0b, 1,055 messages) was seeded from a `/tmp` copy into devCoder, then streamed 180-character pushes into one answer. The harness patched the served `useStickToBottom.ts` to log every `stickToBottom` flip, and a capture-phase scroll listener logged the last scroll events and pins. A run counts as failed when the view ends more than 80 px from the bottom, which means it stopped following the stream.
+
+The original 6 × 250-push soak trims only a few times per run, so it reproduces rarely: 2 of 13 runs on the first tries, then 0 of 12. Each trim is one chance for the race, so the rate matrix uses 30 short streams (40 pushes each). The 12 runs mix push intervals of 16 and 30 ms, 4× CPU throttling, a 900×600 window, a synthetic seed, and a switch away and back mid-stream.
+
+| | Runs that lost the pin | `stickToBottom` flips |
+|---|---|---|
+| Before | **9 / 24** | 9, each a single scroll 97–156 px from the bottom |
+| After | **0 / 24** | 0 |
+
+## Root cause
+
+Every flip had the same trace:
+
+```
+pin       st 28783  sh 29476            ← ResizeObserver pin, stuck
+scrollev  st 28783  sh 29476  first=b036…
+scrollev  st 15136  sh 16000  first=ed091…   → stickToBottom = false (distance 171)
+```
+
+1. A streamed push pushes the window over `TRANSCRIPT_CHAR_BUDGET`, so the render advances `windowStart` and the oldest entry unmounts (`first` changes, scrollHeight drops ~13k px).
+2. With `overflow-anchor: none`, the next layout **clamps** scrollTop to the shorter body. A clamp is a scroll, so the browser queues a scroll event for the following frame.
+3. Before that event is dispatched, the next push commits and grows the streaming answer by a few lines.
+4. `onBodyScroll` measures the distance at dispatch time (171 px > `STICK_BOTTOM_PX`) and takes the clamp for the user scrolling up. `pinning` doesn't cover it, because the clamp is not one of our pins.
+5. Unstuck, the render stops advancing the window (`stickToBottom.current ? Math.max(…) : windowStart`), and the ResizeObserver stops pinning. Nothing re-sticks without a user scroll, so the window grows for the rest of the session.
+
+Ruled out by the traces: the #607 forceStick path, #1489's tail-first fill-in, and ResizeObserver pins. Every pin's scroll event landed inside its `pinning` window. Reading `stickToBottom.current` during render is only one render stale, so on its own it can't stop trimming for good.
+
+## Fix
+
+`useStickToBottom`: when `start` advances while stuck, `onBodyScroll` ignores scroll events for two frames. The clamp happens in the next frame's layout, and its event is dispatched in the frame after, before that frame's rAF callbacks. The ResizeObserver then pins as usual. The fix adds no layout read: a trim can come with any streamed push, and #1489's "one layout read per tail-only phase" test still holds. A user scroll-up within those two frames is dropped, and the next scroll event unsticks.
+
+An earlier version forced layout in the effect so the clamp would happen inside a `pinning` window. That broke #1489's no-layout-read-per-push test, so it was dropped.
+
+## Regression test
+
+`test/threadView.test.tsx`, "stays stuck when a trim's scrollTop clamp lands after the tail grew". It holds rAF callbacks until a simulated frame runs, then appends to trigger a trim. It clamps scrollTop, grows the tail 200 px, and dispatches the clamp's scroll event. It then asserts that the next append still trims. It fails on the old hook.

electron/store.js

@@ -103,9 +103,15 @@ const ENVELOPE_THROTTLE_MS = 2000;
  * 3–5 MB of heap once hydrated, and browsing used to keep every one for the
  * session. Past this count the least recently read clean, non-running
  * thread drops back to its shard and is re-parsed on the next read.
- * ponytail: count cap, not bytes; 8 × the largest real shard is ~40 MB.
  */
 const MAX_HYDRATED_THREADS = 8;
+/**
+ * Byte budget for the same LRU, in shard JSON chars (parsed heap runs
+ * ~1.3× that). Eight 3–4 MB real threads held ~32 MB of heap under the
+ * count cap alone. Soft: dirty, working and in-flight threads stay, and the
+ * most recent read always stays so one huge thread is not re-parsed per read.
+ */
+const MAX_HYDRATED_BYTES = 16 * 1024 * 1024;
 
 /** Per-thread transcript files live next to coder-store.json (#225). */
 const MESSAGES_DIR = "messages";
@@ -312,7 +318,8 @@ class Store {
     // _messagesRaw; still-lazy blob slices on _messagesLazy (legacy envelope
     // only, dropped after the shard split).
     this._messagesHydrated = {};
-    // Read order of hydrated ids, oldest first (LRU for _evictHydrated).
+    // Read order of hydrated ids, oldest first (LRU for _evictHydrated),
+    // mapped to the transcript's JSON length when last parsed or flushed.
     this._hydratedReads = new Map();
     this._messagesLazy = null;
     this._messagesRaw = new Map();
@@ -726,6 +733,8 @@ class Store {
       let json = null;
       if (Object.prototype.hasOwnProperty.call(this._messagesHydrated, id)) {
         json = JSON.stringify(this._messagesHydrated[id] || []);
+        // Refresh the LRU size estimate in place (Map.set keeps the order).
+        if (this._hydratedReads.has(id)) this._hydratedReads.set(id, json.length);
       } else if (this._messagesRaw.has(id)) {
         json = this._messagesRaw.get(id);
       } else {
@@ -1276,8 +1285,7 @@ class Store {
    */
   _hydrateMessages(threadId) {
     if (Object.prototype.hasOwnProperty.call(this._messagesHydrated, threadId)) {
-      this._hydratedReads.delete(threadId);
-      this._hydratedReads.set(threadId, true);
+      this._touchHydrated(threadId);
       return this._messagesHydrated[threadId];
     }
     let json = null;
@@ -1310,21 +1318,36 @@ class Store {
     }
     this._messagesHydrated[threadId] = val;
     this._messagesRaw.delete(threadId);
-    this._hydratedReads.set(threadId, true);
+    this._hydratedReads.set(threadId, json.length);
     this._evictHydrated();
     return val;
   }
 
   /**
-   * Drop least recently read transcripts past MAX_HYDRATED_THREADS back to
-   * their shard. Only a thread whose shard on disk is current is eligible:
-   * never one that is working, dirty, mid-flush, or still on the legacy blob.
+   * Move a hydrated id to the most recent end of the LRU, keeping its size.
+   * @param {string} threadId
+   */
+  _touchHydrated(threadId) {
+    const bytes = this._hydratedReads.get(threadId) || 0;
+    this._hydratedReads.delete(threadId);
+    this._hydratedReads.set(threadId, bytes);
+  }
+
+  /**
+   * Drop least recently read transcripts past MAX_HYDRATED_THREADS or
+   * MAX_HYDRATED_BYTES back to their shard. Only a thread whose shard on disk
+   * is current is eligible: never one that is working, dirty, mid-flush, or
+   * still on the legacy blob, and never the most recent read.
    */
   _evictHydrated() {
     const hydrated = this._messagesHydrated;
     const ids = Object.keys(hydrated);
     let excess = ids.length - MAX_HYDRATED_THREADS;
-    if (excess <= 0) return;
+    let bytes = 0;
+    for (const id of ids) bytes += this._hydratedReads.get(id) || 0;
+    if (excess <= 0 && bytes <= MAX_HYDRATED_BYTES) return;
+    let newest;
+    for (const id of this._hydratedReads.keys()) newest = id;
     const working = new Set(
       (this.data ? this.data.threads : [])
         .filter((t) => t && t.status === "working")
@@ -1337,8 +1360,9 @@ class Store {
       ...this._hydratedReads.keys(),
     ];
     for (const id of order) {
-      if (excess <= 0) break;
+      if (excess <= 0 && bytes <= MAX_HYDRATED_BYTES) break;
       if (
+        id === newest ||
         !Object.prototype.hasOwnProperty.call(hydrated, id) ||
         working.has(id) ||
         !this._messageShards.has(id) ||
@@ -1349,6 +1373,7 @@ class Store {
       ) {
         continue;
       }
+      bytes -= this._hydratedReads.get(id) || 0;
       delete hydrated[id];
       this._hydratedReads.delete(id);
       excess -= 1;
@@ -2036,8 +2061,7 @@ class Store {
       `Older messages were dropped to cap this transcript at ${MAX_MESSAGES_PER_THREAD}.`,
     );
     this._markMessagesDirty(threadId);
-    this._hydratedReads.delete(threadId);
-    this._hydratedReads.set(threadId, true);
+    this._touchHydrated(threadId);
   }
 
   /**
@@ -2526,4 +2550,5 @@ module.exports = {
   SAVE_DEBOUNCE_MAX_MS,
   ENVELOPE_THROTTLE_MS,
   MAX_HYDRATED_THREADS,
+  MAX_HYDRATED_BYTES,
 };