Skip to content

fix(windows): support long paths in scan tooling - #435

Open
faizan-oai wants to merge 43 commits into
mainfrom
agent/support-windows-long-path-inventories
Open

faizan-oai wants to merge 43 commits into
mainfrom
agent/support-windows-long-path-inventories

Conversation

@faizan-oai

@faizan-oai faizan-oai commented Aug 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Windows scans failed when repository paths, scan state, generated artifact names, or Deep Scan directories exceeded legacy Win32 path limits. Filesystem operations now use extended-length Windows paths while persisted state and public scan results retain ordinary portable paths.

Integration with current main is on hold. The original merge trial against 173483d7 produced 13 conflicts, including target registration and workbench storage. At dd156058 there are 14. Main has already removed the optional dependency chain that the branch updates in its eval lockfile.

Changes

  • Add shared, UTF-16-aware helpers for Windows filesystem paths and portable path spelling.

  • Support long paths throughout inventory generation, candidate normalization and ranking, project configuration, security-policy resolution, scan finalization, exports, and workbench state.

  • Preserve historical scans and target identities across ordinary and legacy extended path spellings without rewriting sealed scan history.

  • Normalize complete Deep Scan artifact, heartbeat, ledger, publication, backup, and generated scan-directory paths.

  • Integrate the then-current main and its canonical plugin source layout, publication module, case-insensitive Windows history queries, scope handling, and publication-copy recovery.

  • Compare portable parent paths when archiving a scan whose longer archive name crosses the Windows path threshold. Add native Windows coverage for that boundary and for legacy scan-root aliases.

  • Derive the Windows long-filename fixture from its actual temporary directory, so it still exceeds 260 UTF-16 code units under short CI roots.

  • Exercise real Windows long-path behavior using bundled ripgrep, and exclude generated Python bytecode from published packages. Keep the scope parser test isolated from the real filesystem when the shared helper constructs paths.

  • Preserve the ordinary SQLite filename when an otherwise equivalent drive or UNC prefix alone exceeds the default pager budget. Filesystem metadata still uses the extended path.

  • Preserve the identity of a verbatim-only child during archive relocation and recovery, including when a distinct ordinary sibling exists.

  • Add a native Windows workbench write and read at the SQLite filename limit. Update the old eval lockfile within its existing dependency ranges and carry over the Ubuntu profile setup from #2418 for branch CI.

Testing

Earlier branch revisions:

  • Full Python source and plugin suite: 1,087 passed, 5 skipped, and 104 subtests passed.
  • Full Bun 1.3.14 seeded (12345) and randomized SDK suites at merge commit 9c699ae: 2,111 passed, 44 skipped, zero failures each.
  • After aligning the Windows scope probe with the shared helper, its two focused tests and formatting check passed.
  • Focused runtime and scan-root alias tests: 142 passed, 23 platform skips, zero failures. Native Windows cases are skipped locally and run in hosted Windows CI.
  • SDK/MCP typechecks, generated-model checks, full SDK formatting, Ruff 0.16.1, portable plugin source checks, and git diff --check passed.
  • Frozen SDK and MCP installs passed with pnpm 11.9.0.
  • Hosted Linux, macOS, Windows, installed-package, and container checks run on the updated commit.

On the first repair head, the path and archive suites passed on Linux (73 passed, four Windows-only skips). A real SQLite pager on Linux with simulated Win32 path handling opened the 1,030-byte drive and UNC filenames read-only and writable. The original branch exited before the OS open. The first full branch run also identified an older eval lockfile audit failure and Ubuntu 26.04 AppArmor profile lookup failure.

On dd156058, the native Windows Server 2025 / Python 3.12 job passed 482 tests and 40 subtests with zero skips, failures or errors. The JUnit report records a successful workbench CLI write and read at a 1,030-byte ordinary drive filename and recovery of coverage.json. after archiving with coverage.json present. UNC and enforcement of legacy Windows policy were simulated separately.

The dependency update preserves the optional eval features. A frozen install and the complete deterministic eval suite passed. With the same Trivy 0.76.1 database, the 724-component eval inventory went from two HIGH findings to zero HIGH or CRITICAL findings. Bash syntax and both AppArmor profile layouts, missing profiles, and preference when both exist were checked. Ruff lint and format, the SDK CI build, plugin source compatibility and its nine Node tests passed. The full CI dispatch passed all 65 jobs on this branch head, including the Ubuntu installed-package job. The PR conflicts with main, so the integrated result still needs CI.

Risk and rollout

Extended-length spelling is limited to Windows filesystem access. Persisted paths, scan metadata, target identities, completed artifacts, containment checks, and non-Windows behavior need revalidation against current main. The existing test rejects verbatim-only target directories. Main accepts those directories at target validation and independently checks repository generation before reusing a target ID. Preserve generation ownership and decide the intended directory compatibility before landing. The receipt-specific fix is reviewed separately in #2386. The hosted Windows job covers a local drive. A native UNC share and a Windows host enforcing legacy path policy remain unverified.

Public disclosure review

The checklist below was completed for earlier work. The incremental repair review covered the branch, title, updated description, new commits, diff, synthetic test data and public references.

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

@github-actions github-actions Bot added the bug Something isn't working label Aug 14, 2026
@mldangelo-oai

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 054f72a62a

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

if os.path.normcase(resolved) != os.path.normcase(candidate) or not candidate.is_file():
return None

P1 Badge Compare portable artifact paths before rejecting them

On Windows, when an artifact path reaches the 248-character threshold, filesystem_path(candidate) adds the \\?\ prefix, while the persisted candidate passed by scan_result remains portable. This raw comparison therefore rejects the valid artifact (and candidate.is_file() may also fail on its unprefixed long spelling), causing completed scans to omit reports, findings, and SARIF from their returned artifact map. Normalize both paths with portable_path and inspect the resolved filesystem path instead.

AGENTS.md reference: sdk/typescript/AGENTS.md:L23-L24

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: b4d2f045f7

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b4d2f045f7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 6dc0fd8cca

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@mldangelo-oai

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6dc0fd8cca

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 9a7f97c7f1

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9a7f97c7f1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 1fa404ff11

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 99e90339db

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Collaborator

@codex review

The Windows inventory regression now gets native separators when it reopens Git and ripgrep paths. Please check the change at 3f0a13e. The plugin typecheck passed and the Windows SDK job has restarted.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: 3f0a13eed4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Collaborator

@codex review

The telemetry PR landed on main. Its receipt-directory import is now integrated in 6ad5be8 alongside the Windows path handling. Please verify that registration still reports the portable directory while receipt reads use the full filesystem path.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 6ad5be837f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Collaborator

@codex review

9fa7731 adds the Windows path helper to the compressed-usage test fixture's file list. The six failing SDK regression cases now pass locally on the integrated branch.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9fa7731e22

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Collaborator

@codex review

1c153c9 addresses the saved-worker recovery comment, including the reducer and coverage-receipt consumers. Could you check the follow-up? The regression tests cover both directions of the stored-path spelling mismatch.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 1c153c9b19

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Collaborator

@codex review

The last conflict with main was the scan-directory error path in the finalizer. In 43d56fa it keeps the new diagnostic context and the Windows canonical-directory check. All 188 applicable finalizer and Windows path tests passed locally.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: 43d56fa8c7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛡️ Codex Security Review · Automatically triggered

Here are some automated security review suggestions for this pull request.

Reviewed commit: 732a69abf2

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Copy link
Copy Markdown
Collaborator

@codex review

Please check that long Windows paths and rename-source fallback both work in the updated inventory code. Updated head: af727e0b54.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: af727e0b54

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai mldangelo-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please hold this branch while reconciling it with current main. I confirmed two path regressions at af727e0.

  • database_path() prefixes the filename before SQLite can select its spelling. With a 1,030-byte ordinary filename, the receipt command handed SQLite 1,034 bytes for a drive path and 1,036 for UNC. Both exited 73 before the file-open callback. Main returned the seeded receipt for both. This used the real SQLite pager on Linux with the default Win32 pathname budget simulated. Preserve the ordinary spelling through filename selection, including for writable opens, and carry the byte-boundary coverage from #2386.
  • The archive relocation callback joins the relative suffix to an ordinary archive root. For a stored Windows worker result with a verbatim-only leaf such as result.json., the rebased string loses the namespace that identifies that file. This was reproduced lexically with the branch's callback. Rebase under the extended archive root, then convert the complete result to ordinary spelling only if it still denotes the same object. Keep a regression for an ordinary sibling with the trailing dot removed.

A merge with current main conflicts in 13 files, including the target registration and inventory code. Preserve main’s checkout-generation ownership checks and Windows output-directory protections when porting the changes. The new rejection of existing verbatim-only target directories also needs an explicit compatibility decision or a narrower fix. The earlier green CI predates this integration.

Copy link
Copy Markdown
Collaborator

#2689 ports the writable workbench database and scan archive cases from this PR onto current main. It covers drive and UNC filename selection for SQLite, matching an existing scan across ordinary and extended path spellings, and preserving a verbatim trailing-dot file during archival. It keeps the checkout and workbench state protections on main. The broader inventory and target-registration changes remain here. #2386 covers read-only finding receipt inspection independently.

All seven new native cases passed on Windows Server 2025, including CLI writes and reads at a 1,030-byte drive filename through both input spellings. The complete Source CI run passed. UNC paths and legacy policy were exercised in simulation. Windows JUnit report

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working platform:windows Windows, PowerShell, Windows paths, or Windows ACLs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants