Repository navigation
🌱 Move definitions to internal/checkdef - #5427
Merged
justaugustus merged 2 commits intoNov 6, 2026
Merged
Conversation
Each check's collector is wired twice: once in the check body and once in probe mode's 20-case switch, with nothing keeping them in step. ossf#5425 moved Maintained and CI-Tests to check definitions. This change plans the other 18 checks, has probe mode read the same definitions, and builds the registry from them, with no observable change. The deltas record two current behaviors as tested requirements: checks and probes collect the same raw data (Packaging excepted), and every check a registered probe needs is a registered check. Assisted-by: LLM Signed-off-by: Stephen Augustus <foo@auggie.dev>
Probe mode in `pkg/scorecard` must read the same definitions that the checks run from. An internal package lets both import them without adding exported API. The exported `checks.Maintained` and `checks.CITests` stay as wrappers, and their differential tests pass unchanged. Assisted-by: LLM Signed-off-by: Stephen Augustus <foo@auggie.dev>
justaugustus
temporarily deployed
to
integration-test
November 6, 2026 11:06 — with
GitHub Actions
Inactive
justaugustus
marked this pull request as ready for review
November 6, 2026 11:09
justaugustus
requested review from
spencerschrock
and removed request for
a team
November 6, 2026 11:09
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5427 +/- ##
===========================================
+ Coverage 66.80% 79.68% +12.87%
===========================================
Files 230 264 +34
Lines 16602 14900 -1702
===========================================
+ Hits 11091 11873 +782
+ Misses 4808 3026 -1782
+ Partials 703 1 -702 🚀 New features to boost your workflow:
|
internal/checkdefinternal/checkdef
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
An OpenSpec proposal plus the first step of the refactor it plans (infra/tests/other). No change to scores, findings,
or output.
What is the current behavior?
Each check's wiring is written twice: once in the check's own function, and again in probe mode's 20-case
assignRawDataswitch, which carries anolint:gocognit,gocyclo. Nothing keeps the two in step. #5425 added checkdefinitions and moved Maintained and CI-Tests onto them, with differential tests against the old bodies.
What is the new behavior (if this is a feature change)?
Behavior is unchanged. This PR has two commits:
openspec/changes/consolidate-check-definitions/plans the rest of the work. The other 18 checks move todefinitions in batches. Probe mode reads the same definitions, so
assignRawDatagoes. The check registry(names, request types, experimental gating) is built from the definitions. Consistency tests guarantee one
definition per check and a collector for every check a probe needs. The spec deltas record two current
behaviors as tested requirements: checks and probes collect the same raw data, and every check a registered
probe needs is a registered check.
checkstointernal/checkdef, sopkg/scorecardcan read it later without new exported API. A check runs ascheckdef.Maintained.Run(c). Theexported
checks.Maintainedandchecks.CITestskeep their signatures.Step 1 is a move. The differential tests from #5425 cover it unchanged.
Which issue(s) this PR fixes
NONE
Special notes for your reviewer
Main question: is this the plan you want before more checks move? The decisions most worth a second opinion are
in
design.md:returns empty data on other hosts. Probe mode supports only GitHub and GitLab and errors elsewhere. Phase 2 keeps
both behaviors through a separate probe-mode collector. Unifying them would change
--probesoutput, so it's leftfor its own change.
checks. An accessor would bepermanent public API for an implementation detail.
GetAllandGetAllWithExperimentalkeep their signatures andresults, guarded by a snapshot test taken before the switch.
Out of scope: the CI-Tests output order and the duplicate check-name constants noted in #5425. Both stay as they
are; the plan adds a consistency test that pins the constants.
Verification beyond CI:
openspec validate consolidate-check-definitions --strictpasses.TestCITestsDefinitionMatchesLegacyfailed.That shows the differential tests exercise the moved code.
Rollback: revert the second commit. The first commit adds only the OpenSpec change.
Does this PR introduce a user-facing change?