Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

[performanceautofiler]

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	44d28bf5b6c8b448ac4f19638b90cdbdba6be5bc
Compare	6ff57f1935d1cb3396674bec26268a301e5c0628
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Email_IsNotMatch - Duration of single invocation	334.65 ns	587.91 ns	1.76	0.03	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 587.9140948889302 > 353.26904849015745.
IsChangePoint: Marked as a change because one of 11/17/2021 2:42:53 PM, 11/23/2021 3:51:55 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 3:51:55 AM.
IsRegressionStdDev: Marked as regression because -70.5828741771422 (T) = (0 -583.6719188293049) / Math.Sqrt((20.548479861772265 / (33)) + (225.46833403751063 / (20))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (20) - 2, .025) and -0.7155392558514462 = (340.2265012814413 - 583.6719188293049) / 340.2265012814413 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	44d28bf5b6c8b448ac4f19638b90cdbdba6be5bc
Compare	9962c10813de695782baa20f1bf0bb2e9810971d
Diff	Diff

Regressions in PerfLabTests.CastingPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IFooFooIsIFoo - Duration of single invocation	218.29 μs	280.65 μs	1.29	0.38	False
ObjInt - Duration of single invocation	249.46 μs	280.64 μs	1.12	0.37	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf*'

Details

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf.IFooFooIsIFoo

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 280.6496565934066 > 261.9432088293651.
IsChangePoint: Marked as a change because one of 10/16/2021 4:18:44 PM, 11/11/2021 11:33:25 PM, 11/17/2021 2:42:53 PM, 11/23/2021 3:51:55 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 3:51:55 AM.
IsRegressionStdDev: Marked as regression because -10.231182091581285 (T) = (0 -308430.1068510492) / Math.Sqrt((331433289.3165045 / (33)) + (444218998.4880154 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.20886285352633047 = (255140.69354628507 - 308430.1068510492) / 255140.69354628507 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.ObjInt

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 280.6436727335165 > 261.94455902777776.
IsChangePoint: Marked as a change because one of 9/22/2021 5:04:00 PM, 10/29/2021 10:34:07 PM, 11/11/2021 11:33:25 PM, 11/17/2021 2:42:53 PM, 11/23/2021 3:51:55 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 3:51:55 AM.
IsRegressionStdDev: Marked as regression because -9.082524798950823 (T) = (0 -308658.5429815293) / Math.Sqrt((441956257.76034564 / (33)) + (461405557.2852367 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.19646245517031016 = (257975.95373570948 - 308658.5429815293) / 257975.95373570948 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[AndyAyersMS]

Suspect the Regex is from #61490.

The others are noise.

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[AndyAyersMS]

cc @stephentoub

[stephentoub]

This is the same issue as #61786 (comment). The pattern contains ([a-zA-Z0-9_\-\.]+)@. Previously that loop would have been made atomic, but because the @ is now being changed to be case-sensitive, the auto-atomicity optimization bails out immediately upon seeing that the set differs in case-sensitivity from the node following it. The solution here is similarly #61048. That will fix this and more.
cc: @joperezr

[AndyAyersMS]

Also seen on x86: dotnet/perf-autofiling-issues#2514

[stephentoub]

Also seen on x86

I'd expect this to regress similarly regardless of architecture.

Tagging subscribers to this area: @eerhardt, @dotnet/area-system-text-regularexpressions
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	44d28bf5b6c8b448ac4f19638b90cdbdba6be5bc
Compare	6ff57f1935d1cb3396674bec26268a301e5c0628
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Email_IsNotMatch - Duration of single invocation	334.65 ns	587.91 ns	1.76	0.03	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 587.9140948889302 > 353.26904849015745.
IsChangePoint: Marked as a change because one of 11/17/2021 2:42:53 PM, 11/23/2021 3:51:55 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 3:51:55 AM.
IsRegressionStdDev: Marked as regression because -70.5828741771422 (T) = (0 -583.6719188293049) / Math.Sqrt((20.548479861772265 / (33)) + (225.46833403751063 / (20))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (20) - 2, .025) and -0.7155392558514462 = (340.2265012814413 - 583.6719188293049) / 340.2265012814413 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	44d28bf5b6c8b448ac4f19638b90cdbdba6be5bc
Compare	9962c10813de695782baa20f1bf0bb2e9810971d
Diff	Diff

Regressions in PerfLabTests.CastingPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IFooFooIsIFoo - Duration of single invocation	218.29 μs	280.65 μs	1.29	0.38	False
ObjInt - Duration of single invocation	249.46 μs	280.64 μs	1.12	0.37	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf*'

Details

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf.IFooFooIsIFoo

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 280.6496565934066 > 261.9432088293651.
IsChangePoint: Marked as a change because one of 10/16/2021 4:18:44 PM, 11/11/2021 11:33:25 PM, 11/17/2021 2:42:53 PM, 11/23/2021 3:51:55 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 3:51:55 AM.
IsRegressionStdDev: Marked as regression because -10.231182091581285 (T) = (0 -308430.1068510492) / Math.Sqrt((331433289.3165045 / (33)) + (444218998.4880154 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.20886285352633047 = (255140.69354628507 - 308430.1068510492) / 255140.69354628507 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.ObjInt

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 280.6436727335165 > 261.94455902777776.
IsChangePoint: Marked as a change because one of 9/22/2021 5:04:00 PM, 10/29/2021 10:34:07 PM, 11/11/2021 11:33:25 PM, 11/17/2021 2:42:53 PM, 11/23/2021 3:51:55 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 3:51:55 AM.
IsRegressionStdDev: Marked as regression because -9.082524798950823 (T) = (0 -308658.5429815293) / Math.Sqrt((441956257.76034564 / (33)) + (461405557.2852367 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.19646245517031016 = (257975.95373570948 - 308658.5429815293) / 257975.95373570948 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	-
Labels:	area-System.Text.RegularExpressions, os-windows, tenet-performance, tenet-performance-benchmarks, arch-x64, untriaged
Milestone:	-