[Perf] Changes at 12/28/2021 6:18:01 PM

[performanceautofiler]

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	d094a34a4d555d3aef434bc87ad3fefe15d83e08
Compare	15e95fc44be0ea444ac145b3cc2d4207fa1a6fef
Diff	Diff

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
InequalityOperatorBenchmark - Duration of single invocation	11.74 ns	13.11 ns	1.12	0.12	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Details

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix4x4.InequalityOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.109460078459284 > 12.288147412937622.
IsChangePoint: Marked as a change because one of 12/28/2021 8:42:38 AM, 1/5/2022 11:37:43 PM falls between 12/28/2021 8:42:38 AM and 1/5/2022 11:37:43 PM.
IsRegressionStdDev: Marked as regression because -19.598191341561044 (T) = (0 -13.018293385362945) / Math.Sqrt((0.01873680514312647 / (8)) + (0.06181210359770353 / (25))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (25) - 2, .025) and -0.11664210781814657 = (11.658429584748447 - 13.018293385362945) / 11.658429584748447 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	d094a34a4d555d3aef434bc87ad3fefe15d83e08
Compare	15e95fc44be0ea444ac145b3cc2d4207fa1a6fef
Diff	Diff

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
InequalityOperatorBenchmark - Duration of single invocation	11.74 ns	13.11 ns	1.12	0.12	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Details

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix4x4.InequalityOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.109460078459284 > 12.288147412937622.
IsChangePoint: Marked as a change because one of 12/28/2021 8:42:38 AM, 1/5/2022 11:37:43 PM falls between 12/28/2021 8:42:38 AM and 1/5/2022 11:37:43 PM.
IsRegressionStdDev: Marked as regression because -19.598191341561044 (T) = (0 -13.018293385362945) / Math.Sqrt((0.01873680514312647 / (8)) + (0.06181210359770353 / (25))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (25) - 2, .025) and -0.11664210781814657 = (11.658429584748447 - 13.018293385362945) / 11.658429584748447 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	-
Labels:	arch-arm64, os-windows, tenet-performance, tenet-performance-benchmarks, area-CodeGen-coreclr, untriaged
Milestone:	-

[kunalspathak]

Mostly from #61030
@echesakov

[echesakov]

Taking a look

[echesakov]

The resulting diffs look fine

diff --git a/base/20869.dasm b/diff/20869.dasm
index 98bf79427da..c182fa087f7 100644
--- a/base/20869.dasm
+++ b/diff/20869.dasm
@@ -24,35 +24,27 @@ G_M11226_IG02:        ; gcrefRegs=0000 {}, byrefRegs=0000 {}, byref
             bl      System.Numerics.Matrix4x4:get_Identity():System.Numerics.Matrix4x4
             add     x8, fp, #144	// [V03 tmp2]
             bl      System.Numerics.Matrix4x4:get_Identity():System.Numerics.Matrix4x4
-            ldp     x0, x1, [fp,#208]	// [V02 tmp1]
-            stp     x0, x1, [fp,#80]	// [V04 tmp3]
-            ldp     x0, x1, [fp,#224]	// [V02 tmp1+0x10]
-            stp     x0, x1, [fp,#96]	// [V04 tmp3+0x10]
-            ldp     x0, x1, [fp,#240]	// [V02 tmp1+0x20]
-            stp     x0, x1, [fp,#112]	// [V04 tmp3+0x20]
-            ldp     x0, x1, [fp,#0xd1ffab1e]	// [V02 tmp1+0x30]
-            stp     x0, x1, [fp,#128]	// [V04 tmp3+0x30]
-            ldp     x0, x1, [fp,#144]	// [V03 tmp2]
-            stp     x0, x1, [fp,#16]	// [V05 tmp4]
-            ldp     x0, x1, [fp,#160]	// [V03 tmp2+0x10]
-            stp     x0, x1, [fp,#32]	// [V05 tmp4+0x10]
-            ldp     x0, x1, [fp,#176]	// [V03 tmp2+0x20]
-            stp     x0, x1, [fp,#48]	// [V05 tmp4+0x20]
-            ldp     x0, x1, [fp,#192]	// [V03 tmp2+0x30]
-            stp     x0, x1, [fp,#64]	// [V05 tmp4+0x30]
+            ldp     q16, q17, [fp,#208]
+            stp     q16, q17, [fp,#80]
+            ldp     q16, q17, [fp,#240]
+            stp     q16, q17, [fp,#112]
+            ldp     q16, q17, [fp,#144]
+            stp     q16, q17, [fp,#16]
+            ldp     q16, q17, [fp,#176]
+            stp     q16, q17, [fp,#48]
             add     x0, fp, #80	// [V04 tmp3]
             ; byrRegs +[x0]
             add     x1, fp, #16	// [V05 tmp4]
             ; byrRegs +[x1]
             bl      System.Numerics.Matrix4x4:op_Inequality()
             ; byrRegs -[x0-x1]
-						;; bbWeight=1    PerfScore 37.00
+						;; bbWeight=1    PerfScore 17.00
 G_M11226_IG03:        ; , epilog, nogc, extend
             ldp     fp, lr, [sp],#0xd1ffab1e
             ret     lr
 						;; bbWeight=1    PerfScore 2.00
 
-; Total bytes of code 108, prolog size 8, PerfScore 51.30, instruction count 27, allocated bytes for code 108 (MethodHash=f11fd425) for method System.Numerics.Tests.Perf_Matrix4x4:InequalityOperatorBenchmark():bool:this
+; Total bytes of code 76, prolog size 8, PerfScore 28.10, instruction count 19, allocated bytes for code 76 (MethodHash=f11fd425) for method System.Numerics.Tests.Perf_Matrix4x4:InequalityOperatorBenchmark():bool:this
 ; ============================================================
 
 Unwind Info:
@@ -63,7 +55,7 @@ Unwind Info:
   E bit             : 0
   X bit             : 0
   Vers              : 0
-  Function Length   : 27 (0x0001b) Actual length = 108 (0x00006c)
+  Function Length   : 19 (0x00013) Actual length = 76 (0x00004c)
   ---- Epilog scopes ----
   ---- Scope 0
   Epilog Start Offset        : 3523193630 (0xd1ffab1e) Actual offset = 3523193630 (0xd1ffab1e) Offset from main function begin = 3523193630 (0xd1ffab1e)

Will try to repro this on Arm64 hardware.