Skip to content

fix(fusion): recheck all acceptance lists after judge repair - #1734

Open
metallbaubehrmann-dev wants to merge 1 commit into
AtomicBot-ai:mainfrom
metallbaubehrmann-dev:fix/fusion-recheck-after-repair
Open

metallbaubehrmann-dev wants to merge 1 commit into
AtomicBot-ai:mainfrom
metallbaubehrmann-dev:fix/fusion-recheck-after-repair

Conversation

@metallbaubehrmann-dev

Copy link
Copy Markdown

Problem

A judge retry can fix task B while breaking task A, which already passed. Fusion reruns all contract checks, but currently applies the new results and judge verdicts only to the retried tasks. A can therefore remain ok with checks.failed: 0 and judge.green: true even when the final contract report says its check failed. If the second check run is unavailable, A can also retain its stale pass.

Change

This is a focused follow-up to #605 on current main, using the native opt-in judge introduced since that PR:

  • Keep worker execution results separate from acceptance evidence.
  • After the existing judge retry, rebuild contract findings/check outcomes for every task from those worker results, then judge every eligible acceptance list against the current files.
  • Use only the post-repair verdicts once a retry has happened. Correct repairs still pass; failed or unavailable checks cannot keep an earlier green verdict.

The worker retry count remains one, judge-off behavior is unchanged, and the additional judge calls use the existing turn budget. No new config flag, worker-runner interface, default policy, or global terminal gate is introduced. This does not port the old 28-file implementation or claim to finish all of #539.

Verification

Linux, Node 25.7.0 (matching the repository/CI version), lockfile install, base 6d1f14fcd4ae08e592a71fe52f0146c5aa7bab64.

  • Before the fix: all four new regression cases fail; the 11 existing judge tests pass.
  • After the fix: the same judge file passes 15/15.
  • Fusion plus affected agent/runtime seam tests: 40 files, 713 passed, 0 failed, 0 skipped.
  • The executable-check case runs real Node processes through verify.run: first [A: pass, B: fail], then [A: fail, B: pass]. Other cases check fresh model evidence, acceptance of a correct repair, and a failed post-repair check runner. Model completions in tests are deterministic doubles, not a model-quality benchmark.
  • Production typecheck, documentation and import-boundary checks pass. The full test-type debt gate passes with its existing 682 diagnostics unchanged and no new allowances. It required a 6 GiB Node heap here after the default 2 GiB run ran out of memory.
  • Separate independent review: approved the frozen three-file diff; independently reran 134/134 affected tests and checked cancellation, the spend ceiling, a previously cancelled sibling and judge unavailability with four additional deterministic counterprobes. Source hashes were unchanged before/after review.

Test scopes overlap and are not additive. GitHub CI and maintainer acceptance are separate from these local checks.

Refs #539, especially the requirement to recheck the whole list after every repair.
Follow-up to #605 and the invitation to submit a focused change against current main.

A repaired task can invalidate a different task's previously passing
checks or judged evidence. Rebuild acceptance results from worker outcomes
and rejudge the full current fan-out after the existing bounded retry.

Cover cross-task regression through real verify.run commands, fresh model
evidence, correct repairs and an unavailable post-repair check runner.

Refs AtomicBot-ai#539
Follow-up to AtomicBot-ai#605

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant