Repository navigation
fix(deep-scan): report cost-limit exhaustion as stopped - #2492
Aditya-linux wants to merge 3 commits into
Conversation
mangeshraut712
left a comment
There was a problem hiding this comment.
Reporting this as a stopped outcome (exit 2, executionSuccessful: false) matches the scan contract's rule that only completed supports a completed-scan conclusion. Two things before this is ready, going by the Testing note.
-
The contract and CLI doc changes here say the stopped bundle preserves validated findings and deferred candidates. But by your own description that only holds with the follow-up commit, and on this head 8/41 Python and 3 SDK cases fail. Could you push the follow-up onto this branch so the docs and the code land together? Otherwise the contract would describe behaviour the merged code doesn't have.
-
See the inline note on the tokenless failure call. It may explain part of those failures.
|
Thanks for the review. I cherry-picked the supplied checkpoint-preservation follow-up onto this branch as |
…mit-status # Conflicts: # plugins/codex-security/scripts/workbench_db.py
|
Also merged current |
|
Continued this work in #2686, preserving the commits from this branch. The follow-up reconciles the workbench index with the sealed findings after an interrupted SQLite write and retains validated findings from explicitly incomplete drafts. The fresh and resumed CLI regressions and the portable plugin checks passed. |
Summary
When Deep Scan finishes discovery and hits its cost limit during validation, a completed result can make an empty report look conclusive. This change records the cost stop as a failed scan and retains the results available at that point. Addresses #2415.
Changes
The workbench seals the parent draft with the measured cost, its warning, validated findings and deferred candidates. SARIF records
executionSuccessful: false. The contributor incorporated the checkpoint-preservation follow-up inafd673885, which keeps a later preservation attempt from replacing an already sealed checkpoint.Testing
On the published head
6b1dc16ee, four focused workbench cases passed for authored findings, unvalidated discovery and empty results. Two additional cases reproduced the recovery issues below. The contributor also reported 34 focused Python cases passing after merging main. Contributor’s test resultsThe plugin source-contract job failed its file-size check when CI tested this branch with a newer base. CI job
Risk and rollout
Two cases still need fixes before rollout. If SQLite fails after the manifest is sealed but before it records the failed status, the workbench still shows a running scan and retrying the cost stop fails. A parent draft containing a validated finding with
complete: falsealso fails to finalize. Consumers should treat an empty stopped scan as inconclusive.Public disclosure review
The current review covered the branch, commits, changes, discussion and linked check results. A full review of historical CI logs and attachments remains open.