Repository navigation
scorecard: collect results only when a results file is requested - #912
Draft
justaugustus wants to merge 1 commit into
Draft
justaugustus wants to merge 1 commit into
justaugustus wants to merge 1 commit into
Conversation
uploadSARIF appends every scanned result to a package-level slice that only WriteResults reads, and only a -once run with -results-file calls WriteResults. A daemon kept every result for the life of the process. Collection also happens before the upload and before change detection records the commit, so a repo whose upload keeps failing, such as a 403 while the App lacks security_events: write, added the same result every cycle. Collect only after main turns collection on, which it does for a -once run with -results-file. Assisted-by: LLM Signed-off-by: Stephen Augustus <foo@auggie.dev>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the change making Scorecard result collection opt-in.
No open issues 🎉
What changed in this PR
This PR makes Scorecard result collection opt-in, preventing daemon runs from retaining scan results indefinitely or performing unnecessary JSON conversion.
Changes:
- Disables collection by default and enables it only for
-onceruns with-results-file. - Preserves collection of scan results when uploads fail.
- Adds regression coverage for repeated failed uploads with collection disabled and a single failed upload with collection enabled.
| File | Description |
|---|---|
pkg/policies/scorecard/sarif.go |
Adds an explicit collection toggle and skips conversion when disabled. |
pkg/policies/scorecard/sarif_results_test.go |
Tests result retention in both collection modes when uploads fail. |
cmd/allstar/main.go |
Enables collection when a one-time run requests a results file. |
Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
uploadSARIFappends every scanned result to a package-level slice, but onlyWriteResultsreads it, and only a-oncerun with-results-filecallsWriteResults. A daemon kept every result for the life of the process. Collection also happens before the upload and before change detection records the commit. When a repo's upload keeps failing (for example a 403 while the App lackssecurity_events: write), the same result was added on every cycle.Collection is now off by default.
mainturns it on only for a-oncerun with-results-file. The check runs before the JSON conversion, so the daemon also skips that work.Verification: the new
sarif_results_test.gocovers both modes. With collection off, 50 failing cycles keep 0 entries. With it on, one cycle keeps 1 entry.deadcodewasn't run locally because the installed binary is too old for Go 1.27.EnableResultsCollectionis called frommain, so CI'sdeadcodestep should pass.Review focus: in a
-oncerun, a result is still collected when its upload fails. That seems right, because the results file records the scan, not the upload. Is that the intended behavior?