Skip to content

ci: split benchstat threshold per-method sensitivity #62

Open

Description

@millerjp

Summary

Current threshold (10% time/op at p≤0.05, any allocs/op regression) is too loose for O(n) methods (Range, Keys, Map, Values — 10% of 6–60 µs is noise-free) and too tight for sub-20 ns microbenchmarks (where CI jitter alone can cross 10%).

Scope

Either:

A. Tune scripts/check-bench-regression.sh to apply a stricter threshold (5–6%) to benchmarks tagged as O(n) and the current 10% to the fast-path ones.
B. Demote single-op benchmarks to advisory (continue-on-error: true) with a tracking issue.

Acceptance criteria

  1. Threshold policy documented in CONTRIBUTING.md §Performance baseline.
  2. 10 consecutive CI runs on unchanged main show zero false-positive failures.

Source: performance-reviewer + go-quality.

Activity

  1. added this to the v1.0.1 milestone on Apr 21, 2026
  2. added
    testingTests: unit, BDD, benchmarks, fuzz
    performancePerformance-related change
    P2Can be deferred
    on Apr 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Can be deferredperformancePerformance-related changetestingTests: unit, BDD, benchmarks, fuzz

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions