Repository navigation
Stop Ubicloud fan-outs overshooting the vCPU limit and owners deleting each other's VMs (GST-2852) - #4430
Merged
Conversation
…g each other's VMs (GST-2852) The runner's quota check now reads usage and books the new VM's vCPUs under one lock per machine, so shards started together see each other instead of all passing on the same reading. A new room command prints how many VMs of a size fit now, so a fan-out can size each batch. When usage can't be read, a UBI_QUOTA_VCPUS ceiling refuses the create instead of letting it through. VM names keep the whole owner id (up to 31 characters) instead of its first 12, so owners sharing a prefix no longer reap each other's VMs (#3748). A VM named under the old cut form is still reaped by the machine that recorded it. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
|
😎 This pull request was merged. |
…T-2852) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Review-Finding: p1/specialist-batched#1 reader=testing Review-Finding: p1/specialist-batched#2 reader=red-team Review-Finding: p1/specialist-batched#3 reader=testing Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…y (GST-2852) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Merged
capy-ai Bot
added a commit
that referenced
this pull request
Nov 17, 2026
- /autoplan's approval page no longer hides a reviewer's product objection behind "keep your direction". (#4431) - Every skill that loads the shared preamble, on every host, now says to ask a question once and to check a file before handing it over. (#4383) - The Captain manuals now name the `Failed or skipped:` and `Also changed:` lines that READY and lane reports carry. (#4432) - gstack's shell locks hold again on Ubuntu 26.04, where `mkdir` reports success when another process won the race (GST-2247). (#4262) - `test/gstack-lock-race.test.ts` runs the old pattern and the helper against a PATH shim that behaves like uutils 0.8.0 `mkdir`, so it runs on any runner image: the old pattern lets two of four racers win, the helper admits exactly one in each of 60 races and never overlaps across take, hold and release loops, and parallel `gstack-config` writers keep every key. (#4262) - Windows Setup E2E now runs when `bin/gstack-lock.sh` changes. (#4433) - Claude Code's skill list now says when to use each gstack skill, so Opus loads /review and /plan-eng-review when a request fits. (#4387) - A pull request's merge head can no longer keep its `/ship` receipt without a full review. (#4365) - `/ship` now counts review rounds per pull request, and the head that merges always gets a full review. (#4360) - /ship's documentation audit no longer accepts a passing result when an earlier line from the same audit said "blocked". (#4434) - /ship's documentation audit no longer spends an attempt when the audit child writes text after its result. (#4285) - The README lists the two new guides, and the install doc counts its hooks right. (#4403) - docs/releases.md now says the stable release cut treats a failed commit status as red and a pending one as still going, as it has since #4257. (#4436) - /ship's `finish --docs-requested` now accepts a docs fix when the audit named the file with a line number. (#4437) - Document the `.gstack/approval-classes.json` schema, `gstack-approval-classes` CLI commands, and owner approval citation format in `docs/ship-policy.md#approval-classes`. (#4355) - `/eval-plan` now runs four result checks instead of leaving them to judgment. (#4384) - The /review coverage-audit eval no longer fails a run that really read the source file when zsh stops a command line at an `echo ====` separator. (#4364) - Sixteen in-repo documentation links reach their section again. (#4394) - gstack agents now open every status report with what failed, what they didn't verify and what else they changed. (#4273) - Two new short guides. (#4380) - A skill you wrote whose contract still lists `check_after_last_edit` keeps the rest of its contract. (#4439) - The ship policy guide now explains how to merge any pull request with `gstack-ship-queue land`, even in a repository with one maintainer. (#4377) - /ship's test step runs a large unit suite on Linux. (#4426) - Every skill that loads the shared preamble, on every host, now says never to print a whole environment, git config or process table. (#4382) - Pull requests from /ship now open with at most three plain lines, and every merge lands as one commit titled with the PR title. (#4438) - `gstack-pr-wait`, `gstack-ci-gate` and the release cut no longer call a commit green while a check on it failed. (#4257) - `gstack-runaway` now says what a thread's cold and empty wakes cost, not just how many there were (GST-2770). (#4385) - The optional Stop hook no longer checks for a test run after the last edit, and /verify asks for it in plain words instead. (#4428) - /ship's test step says when it ran out of time, and you can set its budget. (#4429) - The remote test command no longer puts every file on one command line. (#4429) - The stable release no longer waits on the nightly Windows tests (GST-2815). (#4400) - Ubicloud fan-outs can no longer overshoot the vCPU limit, and owners no longer delete each other's VMs. (#4430) - hono is pinned to 4.13.16, which fixes five advisories the weekly OSV scan found in 4.13.13 (gstack#4406). (#4408) - `package.json` overrides (#4408) - `test/dependency-security.test.ts`: the lockfile floor check now covers `hono >= 4.13.16`. (#4408) - The nightly Windows test run finishes again. (#4409) - A short write-up checklist in ETHOS.md. (#4381) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Eval fan-outs on the shared Ubicloud project can no longer start more VMs than the room left, an unreadable usage now launches nothing, and two owners whose ids share their first 12 characters no longer delete each other's VMs. This serves the README's promise that heavy suites run on Ubicloud without starving merge checks and canaries.
What changed
The runner now books each VM's vCPUs under a lock, refuses when usage can't be read, and keeps the whole owner id.
Before/after as text: The runner now books each VM's vCPUs under a lock, refuses when usage can't be read, and keeps the whole owner id
Why
A second-model review found that shards started together all read the same usage, so at 448 of 464 vCPUs two of them could both pass and reach 480, and that the check let a create through when usage couldn't be read. Owner ids were also cut to 12 characters, so two owners sharing that prefix reaped each other's expired VMs (#3748). Now each machine books its VMs one at a time, a fan-out can ask how many more VMs fit, and VM names carry the whole owner id.
Blast radius
Only scripts/ubicloud/ubi-runner.sh changes behavior; merge checks and canaries set no
UBI_QUOTA_VCPUS, so they skip the preflight exactly as before (read at the[ -n "$QUOTA_MAX" ] || return 0early return in quota_preflight, and the cached-ceiling and no-ceiling creates are exercised in test/ubicloud-runner-verify-quota.test.ts). The copies in gbrain, harness-evals, gbrain-evals and the ubicloud skill keep the old behavior until they sync this file.Summary, tests, review and docs
Summary
Fan-outs could overshoot the 464-vCPU eval limit because the quota check and the create were not atomic, and a failed usage read let the create go ahead.
upnow reads usage and books its VM's vCPUs under one lock per machine (flock through python3, which also works on macOS, and which ends with the process); a booking counts until its create is answered, and bookings from dead processes or older than any create can take are dropped.ubi-runner.sh room [-s SIZE]prints how many more VMs fit, so a fan-out driver can size each batch to the room left. WithUBI_QUOTA_VCPUSset and usage unreadable, the create is refused withUBI_QUOTA_UNREADABLE; a cached refusal maximum is the API's own limit, so there the API still decides. VM names carry the whole owner id (up to 31 characters, longer is refused withUBI_OWNER_TOO_LONG),recoveraccepts those ids, and a VM that an older runner copy named under the 12-character cut form still counts as the long owner's when this machine's VM record lists it in the old five-column form. New record rows carry a sixth column,full, so a current 12-character owner's VMs are never adopted by a longer owner sharing the prefix.Test Coverage
All new code paths named below have test coverage, against the fake Ubicloud API (no real VMs):
ups started together at 448 of 464 vCPUs create one VM; the other is refused with the first one's booking named, androomreports 1, 0, then 1 after teardown.UBI_QUOTA_VCPUS(with and without a wait),roomsays unknown, and a cached ceiling or no ceiling still creates.gstackminder1andgstackminder2reap only their own VMs; the cut form is reaped only from an older copy's record row, never from a current 12-character owner's row;recoveraccepts a 13-character owner.Not covered: the lock-busy path (quota_lock's 300-second timeout has no test knob).
Tests ran: ubicloud 37
Pre-Landing Review
Pass 1 (one batched specialist reader) found 3 issues, all fixed:
recoverstill rejected owners longer than 12 characters; a 13-character owner could adopt a current 12-character owner's VMs from the shared record; and booking-pruning paths had no tests. One fix-check found nothing blocking. A second-model adversarial read found nothing blocking and recommended shipping as is.Gate edits: none detected (0 listed, 0 read; patterns: gate scan tagged 0 hunks)
Review notes
Left open, all below the blocking floor:
UBI_QUOTA_WAIT; under a slow API, late shards are refused rather than overshooting.UBI_OWNER_TOO_LONGis checked when the script loads, so it also blocks read-only commands for an owner longer than 31 characters.fcntl(Git Bash), the lock error says "could not take this machine's quota lock" instead of naming the missing module.recover --owner <cut form>writes a six-column row, so a recovered old-form VM isn't reaped by the long owner.Exploratory QA
Smoke checks were the focused fake-API tests above, plus a manual
recoverrun with a 13-character owner (exit 0). No UI.Design Review
No frontend files changed — design review skipped.
Eval Results
No prompt-related files changed — evals skipped.
Review bot comments
No review bot comments.
Plan Completion
Plan completion audit: not run (no plan is bound to this branch and no docs/designs/ file matches). This is a bug fix (Plan: exempt).
Issues
Fixes #3748
Verification Results
No Step 8.1 obligations (no plan).
TODOS
No TODO items completed in this PR.
Documentation
Scope: the diff from 18652cc to the working tree on ubicloud-quota-booking-full-owner (ubi-runner.sh quota lock and booking, the new
roomcommand,UBI_QUOTA_UNREADABLE, full owner ids up to 31 characters), checked against the docs that describe the Ubicloud runner.Result: docs/TESTING_INTERNALS.md and the changelog.d fragment already cover the change. docs/skills.md now names
gstack-ubi roomin the /ubicloud summary. README.md and docs/PROJECT_STRUCTURE.md are still accurate.Debt: ubicloud/SKILL.md.tmpl (the agent-loaded skill) does not mention
room,UBI_QUOTA_UNREADABLEor the 31-character owner limit. Updating it needs a template edit followed by gen:skill-docs, which this audit was not permitted to run.Test plan
bun test test/ubicloud-runner.test.ts test/ubicloud-runner-verify-quota.test.ts: 37 pass, 0 failAgent notes
GST-2852 from HARN-1; gstack#3748. Lane GSTA-575.
REVIEW_LOOP: passes=1 blocking_found=0 deferred=0 noted=2 fixed=3 scope=frozen ledgers_read=3 fix_checks=1 dispatches=1
Gate edits: none detected (0 listed, 0 read; patterns: gate scan tagged 0 hunks)
Approvals
No approvals recorded for branch ubicloud-quota-booking-full-owner.
Plan: exempt: bug fix
Plan check: exempt bug fix
Refs GST-2852