Skip to content

Make gstack's shell locks hold when mkdir reports a race it lost (GST-2247) - #4262

Merged
capy-ai[bot] merged 6 commits into
mainfrom
capy/shell-locks-hold-when-mkdir-lies-gst-2247
Nov 17, 2026
Merged

capy-ai[bot] merged 6 commits into
mainfrom
capy/shell-locks-hold-when-mkdir-lies-gst-2247

Conversation

@garrytan

@garrytan garrytan commented Nov 9, 2026 •

Copy link
Copy Markdown
Owner

gstack's shell locks now hold on Ubuntu 26.04, where mkdir can report success after another process has already created the directory. Every lock now goes through one helper, so the kernel lets exactly one process take it.

Summary

  • A new sourced helper, gstack_lock_take in bin/gstack-lock.sh. It runs mkdir only to make the directory and ignores its exit status. The holder is whoever creates the owner file with the shell's noclobber redirect, which opens it with O_EXCL. It returns 0 when taken, 1 when held, 2 when the directory can't be created, and 3 when the owner file can't be written. On 3 the file and the directory are removed again.
  • All 26 places that used mkdir as a lock now call the helper. Each keeps its own waiting, stale-lock recovery and release.
  • The Claude Code wrapper's copied runtime ships the helper, and bin/gstack-render-claude.sh sources it itself.
  • New tests cover the change. A race test uses a mkdir shim that behaves like the uutils one. A disk-full case runs under ulimit -f 0. A source guard fails on any new lock that trusts a bare mkdir.

Why

Ubuntu 26.04 ships Rust coreutils (uutils) as mkdir, and GitHub's ubuntu-latest does too from 2026-11-09. That mkdir exits 0 even when another process created the directory a moment earlier, so two processes could both think they held a lock. With the real uutils 0.8.0 mkdir, the old pattern let two holders in on 132 of 300 races. The helper let in exactly one on every race. Eight parallel gstack-config set calls lost keys on main in 2 of 5 trials, and in 0 of 5 with this change.

Blast radius

This covers every gstack shell path that takes a lock. Those are the session-start updater, gstack-upgrade, ./setup, gstack-config, the Claude render, leases, freeze, brain sync, the Codex probe and /setup-gbrain. With GNU mkdir the behavior doesn't change, and no caller's stale-lock rules changed.
Status: current. Every authored doc changed since base 18652cc matches the new gstack_lock_take helper, so no edits were needed.

Scope: committed, staged, unstaged and new content on capy/shell-locks-hold-when-mkdir-lies-gst-2247, compared with base 18652cc after the merge of main. Checked against bin/gstack-lock.sh, bin/gstack-upgrade, bin/gstack-session-update and bin/gstack-lease.

Documentation health:

  • USING_GBRAIN_WITH_GSTACK.md: Current. The .setup-gbrain.lock.d row correctly describes the O_EXCL pid-file lock, and the merge with main is clean.
  • docs/capy-install.md: Current. The lease lock description matches gstack-lease.
  • docs/troubleshooting.md: Current. The new #shell-locks section and the .setup-lock recovery note match the helper and the refusal text in gstack-upgrade.
  • setup-gbrain/SKILL.md.tmpl: Current. The lock snippet sources the helper and releases with rm -rf. The generated SKILL.md matches it.
  • changelog.d/capy-shell-locks-hold-when-mkdir-lies-gst-2247.md: Current. Read only; not edited.

Coverage debt: none found. Diagram drift: none, since no diagrams are affected.

Test plan

  • CI free-tests run 42747053716 at efac08c passed: 1660 files, carried by the ship gate.
  • macOS and Windows lanes at efac08c passed in runs 42747053716, 42737904127 and 42737903818.
  • Locally, the gstack-lock-race, gstack-ff-update pin and mkdir-lock-guard tests passed: 42 pass, 0 fail.
Agent notes

Review notes

Below the blocking floor (advisory, informational, test-gap or wording items); recorded here and not fixed in this PR.

  • INFORMATIONAL .github/workflows/windows-setup-e2e.yml:47 setup now hard-requires bin/gstack-lock.sh and exits 1 when it is missing. It is also the single arbiter for every install-path lock on Git Bash: the setup Chromium and deps locks, gstack-config, the install registry, the render lock, settings-hook and session-update. But it is missing from both the paths: trigger list and the install-files PATHS list that decide whether the Windows setup e2e jobs run. Its consumers are all listed, so this PR triggers the jobs. A later PR that changes only the helper, for example its noclobber or MSYS behaviour, would skip the Windows install e2e and be covered only by windows-smoke.

  • INFORMATIONAL bin/gstack-lock.sh:40 Under zsh the new 3> owner-file open prints 'file exists' to stderr when the take loses a real race (rc stays 1); bash, dash and busybox stay quiet.

  • INFORMATIONAL bin/gstack-browser-ensure:89 Empty or missing pid is taken over at once, so a holder between mkdir and its pid write loses the lock (same overlap rate as main).

  • INFORMATIONAL bin/gstack-browser-ensure:91 Loop spins forever on rc 2/3 because continue precedes the tries counter.

  • INFORMATIONAL bin/gstack-developer-profile During an upgrade, old releases' rmdir cannot clear the new takeover and .break locks, which now hold a pid file.

  • Skipped under approval a-efc6d50031d4: test/gstack-lease.test.ts:555. The chmod a-w setup doesn't block writes for root, so the test would fail when run as root.

  • Skipped under approval a-efc6d50031d4: bin/gstack-browser-ensure:89. The wait loop treats return codes 2 and 3 as contention and never reaches its timeout. Return code 2 behaves the same on main.

  • bin/gstack-ff-update must run standalone, so it carries a byte-identical inline copy of gstack_lock_take. A test in test/gstack-ff-update.test.ts fails if the two drift.

  • Five tests that hooked into the old mkdir call now observe the same behavior through the new path: codex-probe-cache, gstack-settings-hook-symlink, gstack-lease, session-update-autostash and setup-gbrain-fixture.

  • 17 test fixtures that copy single bin scripts now also copy bin/gstack-lock.sh.

  • Follow-up after the freeze: windows-setup-e2e.yml paths should list bin/gstack-lock.sh.

APPROVALS_RENDER: redacted (repository visibility public); gstack-approvals list shows the full record

Approvals

Approved: yes (record r-1e0aa319ec92). The full approval record stays private.
Plan: exempt: bug fix (R65)
Plan check: exempt bug fix

Agent notes
{
  "schema_version": 1,
  "pr": 4262,
  "head": "34ab8bc84193264718ce00b297c51ef5514a672f",
  "version": "1.99.5.0",
  "base": "main",
  "gate": {
    "tests": {
      "pass": 1674,
      "fail": 0,
      "skip": 0
    }
  },
  "ci_carried": [
    {
      "lane": "tests",
      "run": 44046307918,
      "attempt": 1,
      "tree": "b87d0da5baf9be1c2b9f7be0b31d20508142504b"
    }
  ],
  "session_kind": "unattended",
  "head_tree": "b87d0da5baf9be1c2b9f7be0b31d20508142504b",
  "product_patch_id": "3491b7ae5bd93899ae9f0a93884d7d8ffaf90805",
  "gstack_version": "1.99.0.0",
  "review_depth": {
    "depth": "full",
    "class": "gate",
    "runtime_lines": 312,
    "rules": "66e5174161eecc2c",
    "base": "18652ccc159970db44a038e71840d7bf6f252209",
    "registry": "18652ccc159970db44a038e71840d7bf6f252209",
    "gate_scan": {
      "tagged": 0,
      "head": "34ab8bc84193264718ce00b297c51ef5514a672f"
    },
    "reasons": [
      "gate-class",
      "risk:auth",
      "risk:secret",
      "risk:security",
      "runtime=312"
    ],
    "risk": [
      "auth",
      "secret",
      "security"
    ],
    "mode": "panel"
  },
  "pregate": "requires-remote cleared @b87d0da5baf9 (8 checks, current for 34ab8bc84193)",
  "requires_remote": [
    "free-tests.yml/cso-macos-launcher: https://github.com/garrytan/gstack/actions/runs/44046307918/attempts/1",
    "free-tests.yml/macos-named-regressions: https://github.com/garrytan/gstack/actions/runs/44046307918/attempts/1",
    "windows-setup-e2e.yml/windows-setup: https://github.com/garrytan/gstack/actions/runs/44046307949/attempts/1",
    "windows-smoke.yml/windows-smoke: https://github.com/garrytan/gstack/actions/runs/44046307910/attempts/1",
    "pregate.json/remote_gate: remote-gate tree=b87d0da5baf9 tests=37356 skipped=105 runs=44046307918/1 profiles=ci-reuse/free-tests heads=34ab8bc84193 verified=2026-11-17T08:58:33.866Z"
  ],
  "issues": {
    "fixes": [],
    "refs": [],
    "flakes": []
  },
  "docs": {
    "status": "current",
    "scope": "README.md, docs/, AGENTS.md, CLAUDE.md, CONTRIBUTING.md, USING_GBRAIN_WITH_GSTACK.md, setup-gbrain/SKILL.md.tmpl against merge base 18652ccc (re-audit after merging main)"
  },
  "review_loop": {
    "passes": 2,
    "blocking_found": 1,
    "deferred": 0,
    "noted": 2,
    "fixed": 1,
    "scope": "frozen",
    "rounds": 2,
    "est_input_tokens": 62803,
    "ledgers_read": 7,
    "dispatches": {
      "specialist-batched": 2,
      "red-team": 2,
      "targeted": 1
    }
  },
  "review_targeted": {
    "classes": [
      "auth",
      "secret"
    ],
    "rules": "8811fd300bf305f0",
    "brief_sha": "7f92a0d0a8319c3c0e076fb3b1a4689c45120b2be2c14dfc7d4cb03aafa9d6a5",
    "product_patch_id": "3491b7ae5bd93899ae9f0a93884d7d8ffaf90805",
    "status": "done",
    "findings": 0,
    "raised_by": "targeted"
  }
}

garrytan and others added 2 commits November 9, 2026 22:56
…-2247)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…ock prose

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@trunk-io

trunk-io Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

😎 This pull request was merged.

garrytan and others added 3 commits November 9, 2026 23:49
…a lock

The Claude Code wrapper's copied runtime now includes bin/gstack-lock.sh, and
bin/gstack-render-claude.sh sources it beside itself, so gstack-config set
works in a copied runtime and the render lock is taken by every caller.
Test fixtures that copy single bin scripts now copy the helper too, and the
mkdir lock guard is registered as a repo-wide docs reader.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
gstack_lock_take created the owner file with a noclobber redirect and, when
that redirect failed, treated an existing file as a competing holder. If the
file was created but the write then failed (disk full, quota, ulimit -f), the
helper found its own empty file, returned 1 and left a lock nobody held;
freeze's state lock has no stale recovery, so it stayed busy until removed by
hand. The helper now tells "opened but write failed" from "already held",
removes its own file and directory, and returns 3. The inline copy in
gstack-ff-update is updated byte for byte, and a new test under ulimit -f 0
fails on the old helper.

Review-Finding: s11/outside-voice#1 reader=outside-voice
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@capy-ai
capy-ai Bot merged commit bcece3f into main Nov 17, 2026
58 checks passed
capy-ai Bot added a commit that referenced this pull request Nov 17, 2026
- /autoplan's approval page no longer hides a reviewer's product objection behind "keep your direction". (#4431)
- Every skill that loads the shared preamble, on every host, now says to ask a question once and to check a file before handing it over. (#4383)
- The Captain manuals now name the `Failed or skipped:` and `Also changed:` lines that READY and lane reports carry. (#4432)
- gstack's shell locks hold again on Ubuntu 26.04, where `mkdir` reports success when another process won the race (GST-2247). (#4262)
- `test/gstack-lock-race.test.ts` runs the old pattern and the helper against a PATH shim that behaves like uutils 0.8.0 `mkdir`, so it runs on any runner image: the old pattern lets two of four racers win, the helper admits exactly one in each of 60 races and never overlaps across take, hold and release loops, and parallel `gstack-config` writers keep every key. (#4262)
- Windows Setup E2E now runs when `bin/gstack-lock.sh` changes. (#4433)
- Claude Code's skill list now says when to use each gstack skill, so Opus loads /review and /plan-eng-review when a request fits. (#4387)
- A pull request's merge head can no longer keep its `/ship` receipt without a full review. (#4365)
- `/ship` now counts review rounds per pull request, and the head that merges always gets a full review. (#4360)
- /ship's documentation audit no longer accepts a passing result when an earlier line from the same audit said "blocked". (#4434)
- /ship's documentation audit no longer spends an attempt when the audit child writes text after its result. (#4285)
- The README lists the two new guides, and the install doc counts its hooks right. (#4403)
- docs/releases.md now says the stable release cut treats a failed commit status as red and a pending one as still going, as it has since #4257. (#4436)
- /ship's `finish --docs-requested` now accepts a docs fix when the audit named the file with a line number. (#4437)
- Document the `.gstack/approval-classes.json` schema, `gstack-approval-classes` CLI commands, and owner approval citation format in `docs/ship-policy.md#approval-classes`. (#4355)
- `/eval-plan` now runs four result checks instead of leaving them to judgment. (#4384)
- The /review coverage-audit eval no longer fails a run that really read the source file when zsh stops a command line at an `echo ====` separator. (#4364)
- Sixteen in-repo documentation links reach their section again. (#4394)
- gstack agents now open every status report with what failed, what they didn't verify and what else they changed. (#4273)
- Two new short guides. (#4380)
- A skill you wrote whose contract still lists `check_after_last_edit` keeps the rest of its contract. (#4439)
- The ship policy guide now explains how to merge any pull request with `gstack-ship-queue land`, even in a repository with one maintainer. (#4377)
- /ship's test step runs a large unit suite on Linux. (#4426)
- Every skill that loads the shared preamble, on every host, now says never to print a whole environment, git config or process table. (#4382)
- Pull requests from /ship now open with at most three plain lines, and every merge lands as one commit titled with the PR title. (#4438)
- `gstack-pr-wait`, `gstack-ci-gate` and the release cut no longer call a commit green while a check on it failed. (#4257)
- `gstack-runaway` now says what a thread's cold and empty wakes cost, not just how many there were (GST-2770). (#4385)
- The optional Stop hook no longer checks for a test run after the last edit, and /verify asks for it in plain words instead. (#4428)
- /ship's test step says when it ran out of time, and you can set its budget. (#4429)
- The remote test command no longer puts every file on one command line. (#4429)
- The stable release no longer waits on the nightly Windows tests (GST-2815). (#4400)
- Ubicloud fan-outs can no longer overshoot the vCPU limit, and owners no longer delete each other's VMs. (#4430)
- hono is pinned to 4.13.16, which fixes five advisories the weekly OSV scan found in 4.13.13 (gstack#4406). (#4408)
- `package.json` overrides (#4408)
- `test/dependency-security.test.ts`: the lockfile floor check now covers `hono >= 4.13.16`. (#4408)
- The nightly Windows test run finishes again. (#4409)
- A short write-up checklist in ETHOS.md. (#4381)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant