Skip to content

🌱 Move definitions to internal/checkdef - #5427

Merged
justaugustus merged 2 commits into
ossf:mainfrom
justaugustus:simplification/phase-2-openspec
Nov 6, 2026
Merged

justaugustus merged 2 commits into
ossf:mainfrom
justaugustus:simplification/phase-2-openspec

Conversation

@justaugustus

Copy link
Copy Markdown
Member

What kind of change does this PR introduce?

An OpenSpec proposal plus the first step of the refactor it plans (infra/tests/other). No change to scores, findings,
or output.

What is the current behavior?

Each check's wiring is written twice: once in the check's own function, and again in probe mode's 20-case
assignRawData switch, which carries a nolint:gocognit,gocyclo. Nothing keeps the two in step. #5425 added check
definitions and moved Maintained and CI-Tests onto them, with differential tests against the old bodies.

What is the new behavior (if this is a feature change)?

Behavior is unchanged. This PR has two commits:

  1. openspec/changes/consolidate-check-definitions/ plans the rest of the work. The other 18 checks move to
    definitions in batches. Probe mode reads the same definitions, so assignRawData goes. The check registry
    (names, request types, experimental gating) is built from the definitions. Consistency tests guarantee one
    definition per check and a collector for every check a probe needs. The spec deltas record two current
    behaviors as tested requirements: checks and probes collect the same raw data, and every check a registered
    probe needs is a registered check.
  2. Step 1 of that plan. The pilot's definition code moves from checks to internal/checkdef, so
    pkg/scorecard can read it later without new exported API. A check runs as checkdef.Maintained.Run(c). The
    exported checks.Maintained and checks.CITests keep their signatures.
  • Tests for the changes have been added (for bug fixes/features)

Step 1 is a move. The differential tests from #5425 cover it unchanged.

Which issue(s) this PR fixes

NONE

Special notes for your reviewer

Main question: is this the plan you want before more checks move? The decisions most worth a second opinion are
in design.md:

  • Packaging stays two-mode (D3). The check merges the GitHub and GitLab collectors on a local directory and
    returns empty data on other hosts. Probe mode supports only GitHub and GitLab and errors elsewhere. Phase 2 keeps
    both behaviors through a separate probe-mode collector. Unifying them would change --probes output, so it's left
    for its own change.
  • Definitions live in an internal package (D1), not behind an exported accessor in checks. An accessor would be
    permanent public API for an implementation detail.
  • The registry moves in this change (D4). GetAll and GetAllWithExperimental keep their signatures and
    results, guarded by a snapshot test taken before the switch.

Out of scope: the CI-Tests output order and the duplicate check-name constants noted in #5425. Both stay as they
are; the plan adds a consistency test that pins the constants.

Verification beyond CI:

  • openspec validate consolidate-check-definitions --strict passes.
  • For step 1, I set CI-Tests' definition to the Maintained name, and TestCITestsDefinitionMatchesLegacy failed.
    That shows the differential tests exercise the moved code.

Rollback: revert the second commit. The first commit adds only the OpenSpec change.

Does this PR introduce a user-facing change?

NONE

Each check's collector is wired twice: once in the check body and once
in probe mode's 20-case switch, with nothing keeping them in step.
ossf#5425 moved Maintained and CI-Tests to check definitions. This change
plans the other 18 checks, has probe mode read the same definitions,
and builds the registry from them, with no observable change.

The deltas record two current behaviors as tested requirements: checks
and probes collect the same raw data (Packaging excepted), and every
check a registered probe needs is a registered check.

Assisted-by: LLM
Signed-off-by: Stephen Augustus <foo@auggie.dev>
Probe mode in `pkg/scorecard` must read the same definitions that the
checks run from. An internal package lets both import them without
adding exported API. The exported `checks.Maintained` and
`checks.CITests` stay as wrappers, and their differential tests pass
unchanged.

Assisted-by: LLM
Signed-off-by: Stephen Augustus <foo@auggie.dev>
@justaugustus
justaugustus marked this pull request as ready for review November 6, 2026 11:09
@justaugustus
justaugustus requested a review from a team as a code owner November 6, 2026 11:09
@justaugustus
justaugustus requested review from spencerschrock and removed request for a team November 6, 2026 11:09
@codecov

codecov Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.68%. Comparing base (353ed60) to head (91d6df4).
⚠️ Report is 468 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff             @@
##             main    #5427       +/-   ##
===========================================
+ Coverage   66.80%   79.68%   +12.87%     
===========================================
  Files         230      264       +34     
  Lines       16602    14900     -1702     
===========================================
+ Hits        11091    11873      +782     
+ Misses       4808     3026     -1782     
+ Partials      703        1      -702     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@justaugustus justaugustus added this to the scorecard-simplification milestone Nov 6, 2026
@justaugustus justaugustus changed the title 🌱 Propose one definition per check and move definitions to internal/checkdef 🌱 Move definitions to internal/checkdef Nov 6, 2026
@justaugustus
justaugustus merged commit d34420e into ossf:main Nov 6, 2026
22 checks passed

This branch was previously deployed

2 inactive deployments
gitlab — 91d6df43 Deployed Nov 6, 2026 by justaugustus via gitlab-integration-trusted #5650
integration-test — 91d6df43 Deployed Nov 6, 2026 by justaugustus via integration-trusted #12956
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant