Repository navigation
Let pull requests that only change human-facing docs skip the test suites (GST-1951) - #4178
Merged
Merged
Conversation
…ites (GST-1951) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
|
😎 This pull request was merged. |
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…ate the workflow contract tests Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
… doc, hold the land policy key until a release reads it, and register two more docs reader tests Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…nd report an undecodable anchor instead of crashing Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…k checker's exit codes Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A pull request that only changes human-facing docs now skips the test suites and runs a quick link and docs check instead, so docs fixes merge in minutes.
What changed
lib/docs-only.ts(CLIbin/gstack-docs-only) decides whether every changed path is human docs. The allowlist is rootREADME.md,docs/**/*.mdandchangelog.d/*.md. A denylist keeps agent instructions, policy docs, templates, sections, pinned results, preregistrations and generated outputs out. Any doc that a script, test, workflow or skill names by path stays out too. Renames check both sides. Any git error means "not docs-only".docs-scopejob infree-tests.ymlandwindows-smoke.ymlruns the base branch's classifier on the merge commit. Suite jobs skip only whendocs-scopesucceeded and said docs-only. A newdocs-lanejob checks links on the lines the PR added and runs the tests that read docs by directory. The requiredfree-testscheck still always runs, passes a docs-only PR only after docs-lane passed, and records no tested tree.lanescheck runs the same docs lane and owes no remote gate. A docs-only CI run carries asnot_owed=docs-onlywhen the local classifier agrees.land.docs_only_skipcan drop the merged-tree test list for a docs-only pin. Land keeps every check when a non-docs file on the base tip names a changed doc. The key isn't turned on in.gstack/ship-policy.jsonyet, because older gstack builds refuse it as unknown.Why
Garry asked for this on 2026-11-08: "For docs changes pls stop doing full CI and merge without tests. If there are really only docs." Docs-only PRs spent 45-51 Free Tests runner-minutes and 6-8 minutes of wall time each on suites their changes can't affect (runs 42417651501, 42244137054). In the 300 PRs merged 2026-11-01..11-08, 2 qualify (#4144, #4081). The saving inside gstack is small. The bigger value is that every crew's /ship and pregate get the class by default.
Blast radius
Every pull request now runs one extra
docs-scopejob infree-tests.ymlandwindows-smoke.yml. Code pull requests run the same suites as before. If the classifier crashes or errors, every suite runs. Pushes to main always run the full suites. Land behavior is unchanged until main's policy turns onland.docs_only_skip.Pre-Landing Review
Pre-Landing Review: 14 issues — 0 auto-fixed, 9 asked (9 fixed, 0 skipped), 0 deferred, 5 noted
Gate edits: 30 listed, 20 read, 2 findings — 2 kept (reasons below), 0 restored, 0 open
.github/workflows/free-tests.ymldocs-scope/docs-lane hunk: kept, it is the feature (suites skip only on a successful docs-only verdict).The full R51 panel ran: specialists (testing, security, state machines), red team, native adversarial, a cross-family outside voice (GPT-6.1 Sol) and the targeted gate reader. Pass 1 ran on c06b33a. Pass 2 re-checked the fixes on c06b33a..f285d09.
Fixed:
land.docs_only_skipas an unknown key, so the key is held out of main's policy.no-stale-gstack-brain-refs,codex-web-search-flag) are now registered inDOCS_READER_TESTS.windows-smokeno longer skips at job level, which would have failed every docs-only land.Noted (below the blocking floor; no test, production or data risk today):
./.github/actions/docs-scopeaction file controls the GitHub check result. gstack's carry still reclassifies locally.artifact_missing.pipefail, so agit difffailure there passes.Test Coverage
Coverage: 68% value-weighted (68% including 0 weakly covered paths), 7 gaps. 0 tests added by the audit (audit only). Gaps: the docs-only-lane reader-test branches, the run_changed and api_error paths in the verifyRun docs-only branch, the run-check docsOnly closure, the two land-direct call sites, and a real-action E2E of the docs-scope composite. The check-doc-links CLI gap has since been closed (test added in a91b6c9).
Exploratory QA
Functional CLI probes of
gstack-docs-onlyon an isolated clone: 7 probes, verdict pass. A referenced doc, an agent-instruction name, an executable doc, a push event and this PR's own diff all refuse. An unreferenced new doc is docs-only and selects the 6 reader tests, and a pull_request run writesdocs_only=true. Limits: the workflow-level behavior is covered by YAML contract tests and CI only.Evals
Not run. The selector fell back to all 113 behavioral probes because
.github/actions/docs-scope/action.ymlis an unknown dependency. That is far over the $10 per-PR cap, and no paid case's reference closure reaches the classifier.TODOS
No TODO items completed in this PR.
Documentation
Status: updated. One factual anchor fix; no blockers.
Scope: base b615592..f285d09 on gst-1951-docs-only-prs (5 commits, 32 files) plus the unstaged test/docs-only.test.ts. Audited the docs-only PR feature (GST-1951/R81) against lib/docs-only.ts, bin/gstack-docs-only, lib/land-docs-only.ts, lib/pregate/docs-only-lane.ts, lib/ci-carry.ts and the free-tests, windows-smoke and docs-scope workflows.
Documentation health:
docs/pregate.md Updated (moved the flaky-exception anchor back to the Owner-granted flaky exception heading; the new Docs-only section had taken it over)
docs/ship-policy.md Current (land.docs_only_skip row matches land-merged-checks validation)
docs/troubleshooting.md Current (docs-only-error entry matches the CLI's error line and exit 2)
capy/skill/gstack/factory.md Current
ship/sections/tests.md(.tmpl) Current (template and generated copy agree; the not-owed wording matches ci-carry)
changelog.d fragment Current (matches the classifier's allowlist, denylist and fail-closed rules)
CHANGELOG.md / TODOS.md Skipped (ship-owned mode)
Documentation coverage:
gstack-docs-only CLI / docs lane reference: docs/pregate.md#docs-only; how-to: partial (CI reuse recipe); tutorial: none; explanation: docs/pregate.md
land.docs_only_skip reference: docs/ship-policy.md and docs/pregate.md; no how-to
Debt: common gaps only (no tutorial). The docs-only feature is reachable from README only through docs/pregate.md.
Diagram drift: none found.
Test plan
bun run test(CI carry): green, run 42544881419, 36641 tests at tree 2dea7989ae5dbun test test/docs-only.test.ts test/docs-only-carry.test.ts test/docs-only-workflow.test.ts test/ci-reuse-workflow-wiring.test.ts test/cso-distribution.test.ts test/windows-native-workflows.test.ts test/windows-path-scope-workflow.test.ts test/autoplan-residuals.test.ts: passbun run typecheck: pass; actionlint with shellcheck on both workflows: passAgent notes
land.docs_only_skipis not in.gstack/ship-policy.json. To turn it on, after the first release that contains this change, add"docs_only_skip": ["bun run scripts/test-free-shards.ts --files-from test/land-merged-checks.txt --wall-timeout 600"]underlandand raisemin_gstack_versionto that release in the same PR. Until then land runs every merged check on docs-only PRs.docs/pregate.mdand the check-doc-links CLI exit-code test (both in a91b6c9) landed after pass 2's readers read tree f60b2b. The review-log rows were persisted at tree 2dea79, which includes them, so those two edits got no separate reviewer read.lib/land-docs-only.tsis its own module to avoid an import cycle (ship-policy->land-merged-checks).lib/ship-queue-land-direct.ts.Approvals
No approvals recorded for branch gst-1951-docs-only-prs.
Plan: approved (record r-4bbb070b2d75)
Plan check: approved (record r-4bbb070b2d75)
Refs GST-1951