Skip to content

Commit a348be8

Browse files
CopilotEgorBo
andauthored
Filter out SPMI metric diffs smaller than ±0.001% (#125684)
## Description SPMI metric diff reporting shows noise as changes even when there are no actual code diffs. The `write_metricdiff_markdown_summary` function used a bare `!=` equality check, so any tiny numeric variance was reported as significant. Inlined an `abs(compute_pct(...)) >= 0.01` threshold check directly in the list comprehension filter in `superpmi.py` to suppress metric diffs below ±0.01%. This is consistent with the existing `is_significant_pct` threshold pattern already used in `write_tpdiff_markdown_summary`. A dummy comment change was also added to `src/coreclr/jit/compiler.cpp` to trigger the SPMI pipeline and validate the fix. <!-- START COPILOT ORIGINAL PROMPT --> <details> <summary>Original prompt</summary> > > ---- > > *This section details on the original issue you should resolve* > > <issue_title>JIT: Metrics show changes even with no-diff changes</issue_title> > <issue_description>E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1339686&view=ms.vss-build-web.run-extensions-tab has no diffs, yet several metrics show up as changing. We probably should increase the expected variance of these metrics so that they don't show up unless they are actually changing. > > cc @EgorBo </issue_description> > > <agent_instructions>Please do not report diffs for SPMI metric diffs if the difference is smaller than +/- 0.01%</agent_instructions> > > ## Comments on the Issue (you are @copilot in this section) > > <comments> > </comments> > </details> <!-- START COPILOT CODING AGENT SUFFIX --> - Fixes #125683 <!-- START COPILOT CODING AGENT TIPS --> --- 💬 Send tasks to Copilot coding agent from [Slack](https://gh.io/cca-slack-docs) and [Teams](https://gh.io/cca-teams-docs) to turn conversations into code. Copilot posts an update in your thread when it's finished. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: EgorBo <523221+EgorBo@users.noreply.github.com> Co-authored-by: Egor Bogatov <egorbo@gmail.com>
1 parent 34dd74d commit a348be8

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

src/coreclr/scripts/superpmi.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3564,7 +3564,7 @@ def fmt_val(v):
35643564
metrics_with_diffs = set()
35653565
for metric in all_metrics:
35663566
significant_diffs = [(mch, base, diff) for (mch, _, base, diff) in metric_diffs
3567-
if metric in base and base[metric] != diff[metric]]
3567+
if metric in base and abs(compute_pct(base[metric], diff[metric])) >= 0.001]
35683568
if not significant_diffs:
35693569
continue
35703570

0 commit comments

Comments
 (0)