Repository navigation
chore(specs): gardener checkbox sync - #999
Merged
Merged
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Deploying mcpproxy-docs with
|
| Latest commit: |
e996b81
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://161e92da.mcpproxy-docs.pages.dev |
| Branch Preview URL: | https://claude-spec-gardener.mcpproxy-docs.pages.dev |
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Dumbris
added a commit
that referenced
this pull request
Aug 31, 2026
…und 3 All four verified against GitHub and the tree before applying. 1. windows-tray: "no Windows-tray PR is open or merged" was false. #74 ("feature: windows tray support") merged 2025-10-23 and the cross-platform Go/systray build still ships on Windows. The claim now scopes to what the epic actually tracks — the native WebView2 replacement — for which no PR exists and native/windows/ is a README placeholder. 2. analytics-dashboard: "can never tick from code" conflated two things. A human can run spec 069's T023 Playwright sweep and tick the box; what can never exist is code evidence for it. Reworded so the distinction is explicit. 3. hygiene-spec-gardener: "three gardener runs ... auto-approved by #842" was wrong. Only #999 carries a github-actions APPROVED review. #824 has none and merged ~24 minutes BEFORE the auto-approve workflow landed; #870 records no approving review either. 4. token-bench-public: calling the MCP-specific third "integrated" overstated completion. ToolRet and LiveMCPTool are static retrieval/token corpora and lap-score is a linter — none is a public agent-loop task suite, which is what the task ultimately owes. Also noted for the record: the linux-arm64 Build Binaries failure on the previous commit is a flaky test, not a regression from this docs-only change. TestServiceStartScanDeepOnRunsSourceResolutionAndPass2 (internal/security/scanner) passes locally with -count=3, and two runs of this same branch four minutes apart — 33360429077 and 33360648947, both docs-only — disagreed on it.
2 of 3 tasks
Dumbris
added a commit
that referenced
this pull request
Aug 31, 2026
…ithub warnings (#1136) * chore(roadmap): truth-sync statuses against code, clear all --check-github warnings An evidence-based re-derivation of every non-done roadmap.yaml leaf. Each flip below was verified against the tree, git/gh state and the rendering sites, then independently re-checked by an adversarial pass; claims that did not survive that pass were dropped rather than applied. Statuses that were wrong in the "already shipped" direction: - upgrade-nudge-delta todo -> done (#1085, closes #1081). The note asserted TODO(spec-079/FR-002) markers exist and that the checker calls only /releases/latest; both are false. internal/updatecheck/delta.go now computes the delta and all six surfaces render it. - upgrade-nudge epic in_progress -> done (all five tasks now done). - analytics-dashboard in_progress -> done. Both children shipped; the default-landing half is live (router '/' -> Dashboard, guarded by dashboard-default-landing.spec.ts). Spec 069's one open task is a local Playwright sweep that leaves no artifact and can never tick from code. - discovery-eval-harness todo -> done. Both halves ship: cmd/scan-eval for security recall/FP, and the eval.yml retrieval-d1 gate plus bench/'s own RecallAtK/NDCGAtK + recall@5 parity gate for discovery quality. - hygiene-spec-gardener in_progress -> done (#824, #870, #999, #842). Its recorded blocker is cleared: check-spec-evidence.py is on main. Statuses that were wrong in the "claimed progress it does not have" direction: - windows-tray and windows-tray-window in_review -> todo. No Windows-tray PR is open or merged and native/windows/ is a README placeholder, so "in review" pointed at nothing. The section header is renamed to match. - planning-hygiene todo -> in_progress and action-log-transparency todo -> in_progress; both already have done children. Notes corrected where they would misdirect the next implementer: - token-bench-harness claimed it absorbs discovery-eval-harness by measuring retrieval recall. Recall is already measured, and bench/ already covers token counting, encoding arms (incl. direct_deferred), break-even and reporting. The note now records what is genuinely open: activity-log session replay, measured first-call success/retry rate, and tokens-per-successful-task. - token-bench-public: the MCP-specific third already ships via spec 083 (ToolRet, LiveMCPBench snapshot, lap-score); only agent-loop suites remain. specs/098-tools-preflight/tasks.md: tick T001, T024, T031, each confirmed against the tree (doc.go states both invariants T001 requires; toolslist_snapshot_test.go is labelled T024 and green; README carries all three FR-017 examples). T025/T027/T032/T033 are left unchecked — they are one-time process acts or did not survive verification. Deliberately NOT changed: no epics were added for shipped specs (084-100). roadmap.yaml is the working set by design and ROADMAP.md already renders a generated per-spec progress table, so absence of an epic is not drift. scripts/gen-roadmap.py --check-github: 3 warnings -> 0 errors, 0 warnings. * fix(roadmap): correct six claims flagged by cross-model review All six findings from the opencode gpt-5.6-sol review were independently re-verified against the tree before being applied; none were false positives. (The reviewer's initial claim that spec-gardener-auto-approve.yml is missing WAS a false positive from its glob tool and it self-corrected — the file exists at .github/workflows/spec-gardener-auto-approve.yml.) 1. Un-tick specs/098 T024. T024 asks for a golden snapshot test proving tools/list is BYTE-IDENTICAL across all three modes. Spec 099 (FR-014/015) later converted that test from a no-delta gate into an ENUMERATED-delta gate (toolsListAllowedDelta permits describe_tool and quarantine_security to move), so at HEAD the test no longer demonstrates the property T024 states. It may well have held on 098's own branch, but that is not checkable from the tree, and an unverifiable tick is exactly what this PR exists to remove. Badge 29/33 -> 28/33. 2. discovery-eval-harness note: drop the "SUPERSEDED — folded into token-bench-harness" framing. It contradicted itself once the epic was marked done — token-bench-harness is still unbuilt, so nothing could have been folded into it. Both halves shipped independently. 3. Same note: the eval.yml retrieval-d1 job is REPORT-ONLY on pull requests (continue-on-error, guarding against npx/uvx fetch flake) and blocks only on the nightly schedule. Promoting it to PR-blocking is still open (MCP-742). 4. Same note: the SC-003 recall@5 = 0.68 +/- 0.05 parity gate is asserted in bench/armindex_test.go, not armindex.go — the latter supplies the production index wiring, not the assertion. 5. token-bench-harness note: the literature-derived retry defaults live in bench/session.go (armRetryRates, research D8). bench/report.go only renders and labels them as estimates. The substantive point stands: retry rates are assumed, not observed. 6. token-bench-public note: "CI-wired" overstated the LiveMCPBench snapshot. bench.yml's regular job passes only -corpus-v2; the snapshot is runnable via -livemcptool but not run in CI, and its loader has no retrieval golden set, so it measures token/scale rather than recall. ToolRet does have its own bench.yml job. gen-roadmap.py --check and --check-github both still clean. * fix(roadmap): tighten two claims from cross-model review round 2 Both findings verified against the tree before applying. 1. The retrieval-D1 blocking scope was stated too narrowly. eval.yml sets continue-on-error only for github.event_name == 'pull_request', and the workflow triggers on pull_request, workflow_dispatch and schedule — so the job is report-only on PRs and BLOCKING on both nightly and manual dispatch runs, not "only on the nightly schedule". The note now says so, and records that the workflow's own inline comment carries the same omission. 2. discovery-eval-harness is now done but still declared depends_on: [token-bench-harness], which is unbuilt — a done node cannot depend on an open one, and the edge contradicted the corrected note saying both halves shipped independently. Edge removed; the id remains a valid depends_on target for anything that needs it. gen-roadmap.py --check and --check-github both clean. * fix(roadmap): scope four over-broad claims from cross-model review round 3 All four verified against GitHub and the tree before applying. 1. windows-tray: "no Windows-tray PR is open or merged" was false. #74 ("feature: windows tray support") merged 2025-10-23 and the cross-platform Go/systray build still ships on Windows. The claim now scopes to what the epic actually tracks — the native WebView2 replacement — for which no PR exists and native/windows/ is a README placeholder. 2. analytics-dashboard: "can never tick from code" conflated two things. A human can run spec 069's T023 Playwright sweep and tick the box; what can never exist is code evidence for it. Reworded so the distinction is explicit. 3. hygiene-spec-gardener: "three gardener runs ... auto-approved by #842" was wrong. Only #999 carries a github-actions APPROVED review. #824 has none and merged ~24 minutes BEFORE the auto-approve workflow landed; #870 records no approving review either. 4. token-bench-public: calling the MCP-specific third "integrated" overstated completion. ToolRet and LiveMCPTool are static retrieval/token corpora and lap-score is a linter — none is a public agent-loop task suite, which is what the task ultimately owes. Also noted for the record: the linux-arm64 Build Binaries failure on the previous commit is a flaky test, not a regression from this docs-only change. TestServiceStartScanDeepOnRunsSourceResolutionAndPass2 (internal/security/scanner) passes locally with -count=3, and two runs of this same branch four minutes apart — 33360429077 and 33360648947, both docs-only — disagreed on it. * fix(roadmap): ToolRet is not MCP-specific (cross-model review round 4) Verified: bench/corpusio/toolret.go documents ToolRet as a general tool-retrieval corpus whose documentation fields are "heterogeneous JSON/text, NOT MCP input schemas" (Schema stays nil). Grouping it under "the MCP-specific third" was wrong. The token-bench-public note now says public corpus/linter groundwork is integrated while NO public agent-loop task suite is, and names the ToolRet caveat explicitly so the next implementer does not re-inherit the error. Round 4 found no other issues; the round-3 windows-tray, analytics T023 and gardener corrections were all confirmed accurate against GitHub and the tree. * fix(roadmap): discovery-eval-harness is in_progress, not done (review round 5) The final review round caught an error in this PR's own flip, and it is the same failure mode the PR was written to fix: an epic marked done while a governing FR is unmet. Spec 065 FR-009 and SC-005 require CI to FAIL on a discovery regression beyond tolerance. The retrieval-D1 job sets continue-on-error for pull_request (D1 depends on npx/uvx fetches, a known flake source), so on the PR path CI does not fail — and eval.yml's own comment records the promotion to PR-blocking as still planned (MCP-742). The HARNESS shipped; the GATE did not. Reverted to in_progress and added the remaining work as a child task (discovery-eval-pr-blocking, MCP-742) so the epic's status has something in the DAG to point at. The reviewer also read CN-002 (frozen corpus, never live) as violated because D1 boots a live mcpproxy serving seven reference servers. Recorded as a weaker tension to adjudicate rather than a violation: #931 pinned all seven upstreams to freeze-era versions and the job gates on the exact corpus ID set, so the corpus is reproducible in practice. Whether that satisfies CN-002 or a committed snapshot is required is a judgement for the spec owner. gen-roadmap.py --check and --check-github both clean (0 errors, 0 warnings).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Description
Automated spec-gardener run (2026-08-17): syncs
specs/*/tasks.mdcheckboxes against real code, per this repo's spec-gardener automation. Applied 26 checkbox ticks, proposing 0 un-tick candidates, across 1 spec with tick candidates surfaced byscripts/check-spec-evidence.py(69 specs scanned: 64 unresolved, 1 removed, 25 relocated-informational, 162 possibly-built). Only- [ ]/- [x]checkbox characters inspecs/*/tasks.mdwere changed, plus a requiredROADMAP.mdregeneration — no task prose, no other files.Note:
claude/spec-gardenerhad been fully merged since the last gardener run (PR #870), so this run restarted the branch from the latestmain(force-with-lease push of already-merged history, per the gardener's merged-PR restart rule) before doing any new work.Testing
git diffthat no other content changed)Applied ticks
All 26 are in
specs/098-tools-preflight/tasks.md(spec 098, "Required-Tools Preflight" — landed via PR #996 / commitf3748106, since the last gardener run). Verified independently by four parallel research passes, each reading the cited source and, where the task promised a test, reading the test body to confirm it actually asserts the behavior (not a stub ort.Skip).internal/server/toolslist_snapshot_test.go:20-24TestToolsListSnapshot_MatchesMergeBaseGoldenspasses for all 3 routing modes, goldens captured from merge-basebfd43e7ceinternal/preflight/reasons.go:88-164reasonTable;reasons_test.go:41-56TestFR003Tableasserts all 15 rows,TestReasonTable_CoversExactlyTheClosedEnumenforces closureinternal/preflight/evaluator.go:36,45,90,97,119-123,183IndexReader/ApprovalReader/StateReader/ConfigPolicy,EvalContext,Evaluate(...) ([]Result, error)returning realerroron infra failureinternal/toolannotations/toolannotations.go:49,internal/server/mcp_annotations.go:273-274ExcludeReasonextracted;mcp_annotations.godelegates viareturn toolannotations.ExcludeReason(...)internal/preflight/classify.go:85-115,120-122ClassifyTooldistinguishesToolClassChangedfromToolClassPendingApproval, honorsAutoApproveToolChangesinternal/preflight/evaluator.go:203-352,307-311,409-416,435evaluateOnewalks FR-004 precedence chain; PendingAuth→oauth_required,server_initializing,sha256/v{N}:{hex}pin check; 657-line table-drivenevaluator_test.gointernal/preflight/scope.go:7-8,100notFoundResultreused for out-of-scope agent-token cases; zeroForProfilecalls anywhere in the new code (grep-confirmed)internal/preflight/suggest.go:29-80Suggest— prefix + Levenshtein ≤2, capped atMaxSuggestions=3; covered bysuggest_test.gointernal/auth/agent_token.go:46,64,internal/httpapi/server.go:536ProfilePinfield propagated through REST auth;auth_profile_pin_test.go(99 lines) covers intersection casesinternal/preflight/contracts_drift_test.go:20-76TestContractsMirrorsTheReasonEnum+TestGeneratedTypeScriptCarriesEveryReasonreads committedfrontend/src/types/contracts.ts;cmd/generate-types/main.go+88 linesinternal/server/preflight_glue.go:44,576-580,30RunPreflightexposed onMCPProxyServer/Server; noserverToolNamesfallback;preflight_glue_test.go:173-221TestRunPreflightPerformsNoUpstreamIOAndNoMutation(instrumented transport + before/after state snapshot)internal/server/tool_gate.go:11-26toolGate/evaluateToolGateconsumed bymcp.gocall_tool variants,mcp_direct_callability.go:182,mcp_code_execution.go:754;preflight_dispatch_parity_test.gohas 10 passing contract testsinternal/storage/activity_models.go:48,internal/runtime/activity_preflight.go:107ActivityTypePreflight;RecordPreflightbypasses the async channel (doc comment confirms);TestRecordPreflightPropagatesWriteFailurepassesinternal/httpapi/preflight.go:200handlePreflight— full validation, tier detection, syncRecordPreflightbefore 200; route atserver.go:866; 4 handler tests cover every 400/503 rule + no-record-on-rejectinternal/httpapi/preflight.go:291,server.go:315,387runPreflightWithWait— 250ms floor, dedicatedpreflightWaitSem(size 4); 3 tests cover deadline, early-termination, semaphore-exhausted degradecmd/mcpproxy/exit_codes.go:58,94,main.go:787-798,cliclient/client.go:2203preflightVerdictError/preflightGeneralError; classifyError dispatch;Client.Preflight; tests incl.TestPreflightGeneralError_NotReclassifiedByStringHeuristicscmd/mcpproxy/tools_cmd.go:844,202newToolsPreflightCmdwith all flags, registered;TestPreflightExitVerdict_WorstClassWins(12>11>10), format/env-var tests,TestToolsPreflightCommand_FlagsAndHelpJSONinternal/preflight/bench_test.go:97-143,internal/httpapi/preflight_bench_test.go:175-213BenchmarkEvaluate*;runPreflightBenchmarkcallshandlePreflightdirectly (includes encoding + activity-record build)server.go:3096-3098,3045,3164,contracts/types.go:294-304enrichServerToolsemitsHashonly whendiscloseHash(operator tier);TestToolHash_NeverDisclosedToAgentToken,TestToolHash_OperatorTierSeesPinmake swagger-verifyreports up to date;go build -tags server ./cmd/mcpproxysucceeds;go test -tags server ./internal/serveredition/... -racepasses;golangci-lint --config .github/.golangci.yml --build-tags serverreports 0 issues on preflight-touching packagescmd/mcpproxy/activity_cmd.go:90,686-768preflightActivitySummary/preflightDetailLines;activity_preflight_test.gocovers rendering + type-filter acceptancefrontend/src/types/api.ts:733,frontend/src/views/Activity.vue:831,637-674'preflight'added toActivityType; filter menu entry + verdict rendering;activity.tsrollup helpersinternal/server/testdata/preflight_sabotage_matrix.json(38 scenarios),internal/server/preflight_e2e_test.goTestPreflightSabotageMatrixE2E/TestPreflightIncidentDiagnosisE2Edrive a real binary + fixture upstreams;assertCellindependently asserts reason/retryable/action/verdict/exit-code +activity list --request-idlookup; siblingpreflight_matrix_test.gosupplies the every-code-appears-once-in-≥1-row reflection checkdocs/api/rest-api.md:616-767POST /api/v1/preflightreference: schema, 15-reason table, 200/400/401/503 rules, dedicatedwait_mssemantics paragraphdocs/cli-management-commands.md:741-838tools preflightsection with exit-code table (0/10/11/12/1) + cron recipe (:801-822) + GitHub Actions recipe (:824-838)docs/features/tools-preflight.md(370 lines, new)Proposed un-ticks (NOT applied)
None. A separate investigation of 7 tasks the checker flagged as UNRESOLVED (ticked, but their cited evidence path never existed in git history) found all 7 were RELOCATED, not actually missing — the code/test lives under a renamed or consolidated file, or (in two cases) the cited path was a templated placeholder / an N/A-with-explanation in the task text itself. None warranted an un-tick candidate:
internal/management/service_impl.gonever existed; code lives ininternal/management/service.go(GetServerTools:957 tagged(T013),TriggerOAuthLogin:993 tagged(T014),checkWriteGates()tagged(T015))(N/A - uses *BoltDB directly, no interface); methods live directly on*BoltDBininternal/storage/bbolt.go:584,623,646, tested inmanager_oauth_test.gointernal/httpapi/server_test.gonever existed; tests live inpatch_server_test.go/server_config_patch_test.gointernal/storage/manager_test.gonever existed; the described test isTestGetRecentSessions_OrdersByLastActivityBeforeTruncationinsessions_filter_test.go:97.json, task text specifies.jsonl; the fixture exists atspecs/090-tray-glance-v2/fixtures/activity-replay.jsonldocs/topic.mdis a templated placeholder describing a doc-link format, not a real target file; that format is used 5+ times inCLAUDE.mdhandleServersChangedinfrontend/src/stores/servers.ts:364-375with an explicit// Spec 047:commentcmd/mcpproxy/teams_register.gowas renamed (not deleted) tocmd/mcpproxy/serveredition_register.goas part of the Teams→ServerEdition rename; still//go:build server, still importsinternal/servereditionDropped by verification
098-tools-preflight T032, 096-batched-call-tools T016, 097-stored-scripts T014 (3 dropped) — all three are "run the full gate suite and confirm green" tasks. GitHub Actions check-runs on the relevant merge commits show
Lint(golangci-lint v2,.github/.golangci.yml),Server Edition,Unit Tests,Build, and (for 098)Verify OpenAPI Artifactsall green — strong partial evidence. However, every one of these tasks explicitly names./scripts/test-api-e2e.shas a required gate, and that script only runs viarelease-qa-gate.yml, aworkflow_call-only reusable workflow invoked exclusively fromprerelease.yml/release.yml— confirmed by grepping.github/workflows/*.ymlfor callers. It does not run on ordinary PR CI. The PR bodies self-report having run it locally, but that's not independently verifiable for these specific commits. Per the tick test's "when uncertain, do not tick" rule, left unticked.098-tools-preflight T001, T024, T025, T027, T031, T033 — not investigated this run: the deterministic checker did not surface them as
possibly_builtcandidates (at least one cited evidence path doesn't exist verbatim for each), so per the gardener's step 3 scope ("judge only the residue the script cannot decide") they weren't in scope for this pass. Left unticked; a future run's checker pass over an updated tasks.md may surface them.7 un-tick candidates across 5 other specs — investigated and dropped (all RELOCATED, not false ticks) — see table above.
Notes
python3 scripts/gen-roadmap.pyin a separate commit after the checkbox-sync commit, per the pre-commit staleness check (2 lines changed: 098-tools-preflight progress percentage).claude/spec-gardenerbranch was reset to the tip ofmainbefore this run (force-with-lease push of already-merged history) because the prior gardener PR (chore(specs): gardener checkbox sync #870) had merged and the branch carried no unmerged work beyond that — per the gardener's merged-PR restart instructions.Cap: 40 applied ticks per run — this run found 26 that survived verification, under the cap.