Skip to content

Add a SARIF exporter for GitHub code scanning#2716

Description

@BILLKISHORE

馃殌 Feature Request

Add a SARIF 2.1.0 exporter alongside the existing JUnit and Hub exporters, so a suite result can be uploaded to GitHub code scanning.

result = await suite.run(agent)
result.to_sarif("giskard.sarif")
- uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: giskard.sarif

馃攬 Motivation

to_junit_xml gets scan results into a CI test report, which answers "did the build pass". SARIF answers a different question: it puts each finding in the GitHub Security tab and as a PR annotation, with a rule, a severity and a fingerprint that keeps an alert stable across runs.

For a scanner that produces security findings rather than test failures, that seems like the more natural surface. It also means a team can gate merges on new findings using the same machinery they already use for CodeQL and dependency alerts.

The tag vocabulary already on scenarios (threat-type:*) maps cleanly onto SARIF rule tags, so findings arrive already categorised.

馃洶 Alternatives

  • Keep JUnit only. Works for pass/fail gating but findings don't reach the Security tab.
  • Leave it to users. The mapping from SuiteResult to SARIF rules and results is non-obvious enough that everyone would rewrite it slightly differently.
  • Put it in the Hub rather than OSS, if surfacing findings is meant to be a Hub concern. Fair, and worth saying so if that's the plan.

馃搸 Additional context

I've written this against main to check it's practical: ~150 lines in checks/export/sarif.py mirroring junit.py, a SuiteResult.to_sarif() method next to to_junit_xml(), and tests. Output validates against the published SARIF 2.1.0 schema. No new dependencies.

Design points I'd want your view on before proposing it:

  1. Only failures and errors become results. Passing checks are omitted, since an alert per passing check would bury the real ones.
  2. Rules keyed by check_kind, so one rule covers every instance rather than one rule per scenario.
  3. Scenario tags become rule tags. This is the part I'd most like a second opinion on, because it's where an OWASP or MITRE mapping would slot in later if you ever wanted one.

Happy to open a PR if this is wanted, or to drop it if exporting findings belongs on the Hub side.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions