Skip to content

fix(history): avoid repeating findings in bulk scan matching - #149

Merged
mldangelo-oai merged 21 commits into
openai:mainfrom
GautamSharma99:fix/bound-scan-history-matching
Nov 17, 2026
Merged

mldangelo-oai merged 21 commits into
openai:mainfrom
GautamSharma99:fix/bound-scan-history-matching

Conversation

@GautamSharma99

@GautamSharma99 GautamSharma99 commented Jul 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #136. Bulk matching returned the same finding evidence once for every comparison involving a scan. The SDK now requests a plan that includes each participating scan’s findings once.

Changes

The SDK requests compact output only for list-unmatched-scan-pairs. The workbench returns a shared findingsByScanId table and chronological comparison batches. The SDK resolves each batch before matching or saving it. An empty findings array is valid. A missing entry reports an error before that batch is matched or saved.

The planner sorts scan IDs once, by start time for bulk matching or completion order for a focused comparison. It reuses the same order to select eligible saved links, including links from scans whose coverage is unavailable.

Testing

A contributor fixture with 12 scans, including one empty scan, produced the same 10 model inputs and 66 comparisons. In that synthetic run the workbench response fell from 5,990,752 to 958,931 bytes. The next ordinary invocation skipped all 66 comparisons.

On a separate synthetic 32-scan history, both formats produced the same 496 pairs. The serialized response fell from 2,119,999 to 141,750 bytes, and the revised planner called the timestamp ordering function 32 times, down from 2,016. Across 84 additional combinations of focus, forced matching, saved links, unavailable coverage and response format, the plans matched the previous PR head exactly.

All five portable plugin checks passed, including nine compatibility tests. Built the bundled plugin with source-matched native artifacts. SDK and MCP type checks, formatting, and git diff --check passed.

The Linux CLI, comparison and runtime suites passed 395 tests with 37 platform skips. The full runtime suite passed on replay with its tools outside the enclosing checkout. All 168 Python scan-history and query tests passed, including subprocess coverage for listing unmatched pairs.

On a 48-scan synthetic history, the compact response preserved all 1,128 pairs and reduced serialized findings from 5,503 entries to 209. Workbench stdout fell from 12,623,327 bytes to 540,800 bytes. The decoded plans were identical.

Ubuntu 26.04 exposed two inventory tests that invoked the multicall env utility under a different name. The PR includes the fixture correction from #2678. Both affected tests passed locally on Linux with the corrected fixture, along with all five portable plugin checks.

Risk and rollout

An older SDK or direct workbench caller still receives inline findings by default. An older plugin ignores the compact request and returns inline findings, which this SDK accepts. An uncached history still lists every eligible pair in one JSON document.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

A historical automated review comment contains an access-restricted report link.

@rajpratham1 rajpratham1 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pull request improves the scalability of scans match --all by replacing bulk loading with paginated retrieval of scan pairs and finding inputs. The implementation introduces bounded page sizes for scan pairs and findings, enforces a maximum serialized input size, validates pagination cursors, and reconciles matches generated across multiple batches before persisting the final comparison. Additional safeguards reject oversized automatic matching results with a clear error, and the workbench CLI and README are updated to document the new behavior and operational limits. Based on the visible changes, the implementation is well-aligned with the stated goal of bounding resource usage while maintaining matching correctness, and no blocking issues are apparent.

@mldangelo-oai

Copy link
Copy Markdown
Collaborator

@codex review exact head 3a1c993

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 3a1c99341f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 3a1c99341f

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@mldangelo-oai mldangelo-oai changed the title fix: bound scans match --all history payloads fix(history): bound scans match --all history payloads Oct 8, 2026
@mldangelo-oai mldangelo-oai added area:findings Finding schemas, identity, deduplication, severity, and comparison decisions. and removed area:findings Finding schemas, identity, deduplication, severity, and comparison decisions. labels Oct 8, 2026
@mldangelo-oai mldangelo-oai changed the title fix(history): bound scans match --all history payloads fix(history)!: avoid repeating findings in bulk scan matching Nov 1, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T06:32:09.292899Z 96137b6 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

mldangelo-oai commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

The integration preserves the older inline workbench response format while sharing the findings used during matching. All 77 Python history tests and 162 SDK tests passed, with three platform skips. The full CI run passed.

I published the maintainer branch in this repository. From the PR branch, you can apply the tested commit with

git fetch https://github.com/openai/codex-security.git maintainer-sync/pr-149
git merge --ff-only FETCH_HEAD

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

Reviewed commit: d72f62113e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Collaborator

This PR branch now contains the tested merge with main, published by fast-forward.

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Chef's kiss.

Reviewed commit: 8c3eb18d77

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai mldangelo-oai changed the title fix(history)!: avoid repeating findings in bulk scan matching fix(history): avoid repeating findings in bulk scan matching Nov 13, 2026
@mldangelo-oai
mldangelo-oai merged commit 95e0c88 into openai:main Nov 17, 2026
66 checks passed
@github-actions github-actions Bot mentioned this pull request Nov 17, 2026
3 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avoid repeatedly materializing quadratic scan-history matching payloads

4 participants