Repository navigation
fix(history): avoid repeating findings in bulk scan matching - #149
Conversation
rajpratham1
left a comment
There was a problem hiding this comment.
This pull request improves the scalability of scans match --all by replacing bulk loading with paginated retrieval of scan pairs and finding inputs. The implementation introduces bounded page sizes for scan pairs and findings, enforces a maximum serialized input size, validates pagination cursors, and reconciles matches generated across multiple batches before persisting the final comparison. Additional safeguards reject oversized automatic matching results with a clear error, and the workbench CLI and README are updated to document the new behavior and operational limits. Based on the visible changes, the implementation is well-aligned with the stated goal of bounding resource usage while maintaining matching correctness, and no blocking issues are apparent.
|
Codex Review: Didn't find any major issues. Delightful! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Security review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
The integration preserves the older inline workbench response format while sharing the findings used during matching. All 77 Python history tests and 162 SDK tests passed, with three platform skips. The full CI run passed. I published the maintainer branch in this repository. From the PR branch, you can apply the tested commit with git fetch https://github.com/openai/codex-security.git maintainer-sync/pr-149
git merge --ff-only FETCH_HEAD@codex review |
|
Codex Review: Didn't find any major issues. 👍 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
This PR branch now contains the tested merge with main, published by fast-forward. @codex review |
|
Codex Review: Didn't find any major issues. Chef's kiss. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Summary
Fixes #136. Bulk matching returned the same finding evidence once for every comparison involving a scan. The SDK now requests a plan that includes each participating scan’s findings once.
Changes
The SDK requests compact output only for
list-unmatched-scan-pairs. The workbench returns a sharedfindingsByScanIdtable and chronological comparison batches. The SDK resolves each batch before matching or saving it. An empty findings array is valid. A missing entry reports an error before that batch is matched or saved.The planner sorts scan IDs once, by start time for bulk matching or completion order for a focused comparison. It reuses the same order to select eligible saved links, including links from scans whose coverage is unavailable.
Testing
A contributor fixture with 12 scans, including one empty scan, produced the same 10 model inputs and 66 comparisons. In that synthetic run the workbench response fell from 5,990,752 to 958,931 bytes. The next ordinary invocation skipped all 66 comparisons.
On a separate synthetic 32-scan history, both formats produced the same 496 pairs. The serialized response fell from 2,119,999 to 141,750 bytes, and the revised planner called the timestamp ordering function 32 times, down from 2,016. Across 84 additional combinations of focus, forced matching, saved links, unavailable coverage and response format, the plans matched the previous PR head exactly.
All five portable plugin checks passed, including nine compatibility tests. Built the bundled plugin with source-matched native artifacts. SDK and MCP type checks, formatting, and
git diff --checkpassed.The Linux CLI, comparison and runtime suites passed 395 tests with 37 platform skips. The full runtime suite passed on replay with its tools outside the enclosing checkout. All 168 Python scan-history and query tests passed, including subprocess coverage for listing unmatched pairs.
On a 48-scan synthetic history, the compact response preserved all 1,128 pairs and reduced serialized findings from 5,503 entries to 209. Workbench stdout fell from 12,623,327 bytes to 540,800 bytes. The decoded plans were identical.
Ubuntu 26.04 exposed two inventory tests that invoked the multicall
envutility under a different name. The PR includes the fixture correction from #2678. Both affected tests passed locally on Linux with the corrected fixture, along with all five portable plugin checks.Risk and rollout
An older SDK or direct workbench caller still receives inline findings by default. An older plugin ignores the compact request and returns inline findings, which this SDK accepts. An uncached history still lists every eligible pair in one JSON document.
Public disclosure review
A historical automated review comment contains an access-restricted report link.