[Perf] Linux/arm64: 6 Regressions on 4/12/2023 8:39:35 PM

### Run Information

Name | Value
-- | --
Architecture | arm64
OS | ubuntu 20.04
Queue | AmpereUbuntu
Baseline | [7a57f6dd800e77508a4ba405e623059935c82f43](https://github.com/dotnet/runtime/commit/7a57f6dd800e77508a4ba405e623059935c82f43)
Compare | [7ce83de7a397cb9331242c178c687307c0e31f88](https://github.com/dotnet/runtime/commit/7ce83de7a397cb9331242c178c687307c0e31f88)
Diff | [Diff](https://github.com/dotnet/runtime/compare/7a57f6dd800e77508a4ba405e623059935c82f43...7ce83de7a397cb9331242c178c687307c0e31f88)
Configs | CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono


### Regressions in System.Collections.Tests.Perf_BitArray

Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL
-- | -- | -- | -- | -- | -- | -- | -- | -- | -- | --
[BitArraySetAll - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size%3a%204).html>) | 8.57 ns | 11.41 ns | 1.33 | 0.06 | False | | |
[BitArrayNot - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Collections.Tests.Perf_BitArray.BitArrayNot(Size%3a%20512).html>) | 307.72 ns | 506.05 ns | 1.64 | 0.01 | False | | |
[BitArraySetAll - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size%3a%20512).html>) | 75.73 ns | 264.19 ns | 3.49 | 0.00 | True | | |

![graph](<https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/refs/heads/main_arm64_ubuntu%2020.04_LLVM%3Dfalse_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/System.Collections.Tests.Perf_BitArray_1.png>)
![graph](<https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/refs/heads/main_arm64_ubuntu%2020.04_LLVM%3Dfalse_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/System.Collections.Tests.Perf_BitArray_2.png>)
[Test Report](<https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/refs/heads/main_arm64_ubuntu%2020.04_LLVM%3Dfalse_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/System.Collections.Tests.Perf_BitArray.html>)

### Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

### Payloads

[Baseline](<https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-a2743b07-c35e-49e2-ac4f-737a65f8a4203336e18679e4429a1/ffb50ae5-893d-4ee5-9749-97babd214708.zip?sv=2021-08-06&se=2023-05-12T02%3A08%3A03Z&sr=c&sp=rl&sig=giC6C7DZGWA33%2FZO9XjA6BvBNrx2Uxs2uTECh1rhNa0%3D>)
[Compare](<https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-06ac10fa-b3a4-4120-b3e5-4ab01db945080e4235248334f2983/82791da7-52cd-43dc-af72-dffec87064f2.zip?sv=2021-08-06&se=2023-05-17T07%3A00%3A49Z&sr=c&sp=rl&sig=2tsWEGM3Op9g3x4ZiStsAdB%2B3eokLITprKSPqSYb0A0%3D>)

<details>
   <summary>Repro Steps</summary>


#### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))
- Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0`
- CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0`
- AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false`
- Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder

Linux
```cmd
# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.Tests.Perf_BitArray* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200
```

Windows
```cmd
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.Tests.Perf_BitArray* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200
```

</details>

<details>


### Payloads

[Baseline](<https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-a2743b07-c35e-49e2-ac4f-737a65f8a4203336e18679e4429a1/ffb50ae5-893d-4ee5-9749-97babd214708.zip?sv=2021-08-06&se=2023-05-12T02%3A08%3A03Z&sr=c&sp=rl&sig=giC6C7DZGWA33%2FZO9XjA6BvBNrx2Uxs2uTECh1rhNa0%3D>)
[Compare](<https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-06ac10fa-b3a4-4120-b3e5-4ab01db945080e4235248334f2983/82791da7-52cd-43dc-af72-dffec87064f2.zip?sv=2021-08-06&se=2023-05-17T07%3A00%3A49Z&sr=c&sp=rl&sig=2tsWEGM3Op9g3x4ZiStsAdB%2B3eokLITprKSPqSYb0A0%3D>)
### Histogram

#### System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 4)

```log

```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.410400606720197 > 9.003586055810645.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 4:36:42 PM falls between 4/9/2023 9:22:58 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -54.65597712738722 (T) = (0 -11.273003138230491) / Math.Sqrt((0.01928771559100608 / (19)) + (0.006453132953245346 / (5))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (5) - 2, .025) and -0.30346203693080487 = (8.648508985174937 - 11.273003138230491) / 8.648508985174937 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```
#### System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512)

```log

```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 506.0492786677301 > 324.281618413946.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 4:36:42 PM falls between 4/9/2023 9:22:58 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -41.4957361772938 (T) = (0 -510.50030481994963) / Math.Sqrt((0.5040844831322225 / (19)) + (141.91330477026344 / (6))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (6) - 2, .025) and -0.6543602869952586 = (308.5786746895072 - 510.50030481994963) / 308.5786746895072 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```
#### System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 512)

```log

```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 264.1859394777212 > 79.27525164298565.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 4:36:42 PM falls between 4/9/2023 9:22:58 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -333.6543282574262 (T) = (0 -264.52996094432234) / Math.Sqrt((0.18189116228039723 / (19)) + (1.8656949061623167 / (6))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (6) - 2, .025) and -2.4975662580193636 = (75.6325803229023 - 264.52996094432234) / 75.6325803229023 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```


### Docs

[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

</details>


---


### Run Information

Name | Value
-- | --
Architecture | arm64
OS | ubuntu 20.04
Queue | AmpereUbuntu
Baseline | [7a57f6dd800e77508a4ba405e623059935c82f43](https://github.com/dotnet/runtime/commit/7a57f6dd800e77508a4ba405e623059935c82f43)
Compare | [7ce83de7a397cb9331242c178c687307c0e31f88](https://github.com/dotnet/runtime/commit/7ce83de7a397cb9331242c178c687307c0e31f88)
Diff | [Diff](https://github.com/dotnet/runtime/compare/7a57f6dd800e77508a4ba405e623059935c82f43...7ce83de7a397cb9331242c178c687307c0e31f88)
Configs | CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono


### Regressions in System.Memory.Span&lt;Int32&gt;

Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL
-- | -- | -- | -- | -- | -- | -- | -- | -- | -- | --
[LastIndexOfValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Memory.Span(Int32).LastIndexOfValue(Size%3a%20512).html>) | 255.68 ns | 812.65 ns | 3.18 | 0.01 | True | | |
[LastIndexOfValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Memory.Span(Int32).LastIndexOfValue(Size%3a%204).html>) | 7.39 ns | 33.86 ns | 4.58 | 0.19 | True | | |
[LastIndexOfValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Memory.Span(Int32).LastIndexOfValue(Size%3a%2033).html>) | 23.45 ns | 85.33 ns | 3.64 | 0.05 | True | | |

![graph](<https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/refs/heads/main_arm64_ubuntu%2020.04_LLVM%3Dfalse_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/System.Memory.Span%28Int32%29_1.png>)
![graph](<https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/refs/heads/main_arm64_ubuntu%2020.04_LLVM%3Dfalse_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/System.Memory.Span%28Int32%29_2.png>)
![graph](<https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/refs/heads/main_arm64_ubuntu%2020.04_LLVM%3Dfalse_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/System.Memory.Span%28Int32%29_3.png>)
[Test Report](<https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/refs/heads/main_arm64_ubuntu%2020.04_LLVM%3Dfalse_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/System.Memory.Span%28Int32%29.html>)

### Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

### Payloads

[Baseline](<https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-a2743b07-c35e-49e2-ac4f-737a65f8a4203336e18679e4429a1/ffb50ae5-893d-4ee5-9749-97babd214708.zip?sv=2021-08-06&se=2023-05-12T02%3A08%3A03Z&sr=c&sp=rl&sig=giC6C7DZGWA33%2FZO9XjA6BvBNrx2Uxs2uTECh1rhNa0%3D>)
[Compare](<https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-06ac10fa-b3a4-4120-b3e5-4ab01db945080e4235248334f2983/82791da7-52cd-43dc-af72-dffec87064f2.zip?sv=2021-08-06&se=2023-05-17T07%3A00%3A49Z&sr=c&sp=rl&sig=2tsWEGM3Op9g3x4ZiStsAdB%2B3eokLITprKSPqSYb0A0%3D>)

<details>
   <summary>Repro Steps</summary>


#### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))
- Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0`
- CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0`
- AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false`
- Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder

Linux
```cmd
# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Span&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200
```

Windows
```cmd
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Span&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200
```

</details>

<details>


### Payloads

[Baseline](<https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-a2743b07-c35e-49e2-ac4f-737a65f8a4203336e18679e4429a1/ffb50ae5-893d-4ee5-9749-97babd214708.zip?sv=2021-08-06&se=2023-05-12T02%3A08%3A03Z&sr=c&sp=rl&sig=giC6C7DZGWA33%2FZO9XjA6BvBNrx2Uxs2uTECh1rhNa0%3D>)
[Compare](<https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-06ac10fa-b3a4-4120-b3e5-4ab01db945080e4235248334f2983/82791da7-52cd-43dc-af72-dffec87064f2.zip?sv=2021-08-06&se=2023-05-17T07%3A00%3A49Z&sr=c&sp=rl&sig=2tsWEGM3Op9g3x4ZiStsAdB%2B3eokLITprKSPqSYb0A0%3D>)
### Histogram

#### System.Memory.Span&lt;Int32&gt;.LastIndexOfValue(Size: 512)

```log

```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 812.647539021511 > 268.2763406584759.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 4:36:42 PM falls between 4/9/2023 9:22:58 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -111.58295939664889 (T) = (0 -820.3781485563792) / Math.Sqrt((0.9974828648898251 / (19)) + (127.6950099114984 / (5))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (5) - 2, .025) and -2.205831976737487 = (255.90179226774762 - 820.3781485563792) / 255.90179226774762 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```
#### System.Memory.Span&lt;Int32&gt;.LastIndexOfValue(Size: 4)

```log

```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.862134232490824 > 7.7988378763488395.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 4:36:42 PM falls between 4/9/2023 9:22:58 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -113.38026955121792 (T) = (0 -33.59967522464688) / Math.Sqrt((0.8445239310581726 / (19)) + (0.04512813770636653 / (6))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (6) - 2, .025) and -3.3340668657423067 = (7.75245889495343 - 33.59967522464688) / 7.75245889495343 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```
#### System.Memory.Span&lt;Int32&gt;.LastIndexOfValue(Size: 33)

```log

```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 85.33435291946611 > 24.807545648300266.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 4:36:42 PM falls between 4/9/2023 9:22:58 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -326.95222695199607 (T) = (0 -85.61347949396098) / Math.Sqrt((0.03715450776281566 / (18)) + (0.20381281976133203 / (6))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (6) - 2, .025) and -2.6353487793516988 = (23.550279406541193 - 85.61347949396098) / 23.550279406541193 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```


### Docs

[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

</details>

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	7ce83de7a397cb9331242c178c687307c0e31f88
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
BitArraySetAll - Duration of single invocation	8.57 ns	11.41 ns	1.33	0.06	False
BitArrayNot - Duration of single invocation	307.72 ns	506.05 ns	1.64	0.01	False
BitArraySetAll - Duration of single invocation	75.73 ns	264.19 ns	3.49	0.00	True

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
LastIndexOfValue - Duration of single invocation	255.68 ns	812.65 ns	3.18	0.01	True
LastIndexOfValue - Duration of single invocation	7.39 ns	33.86 ns	4.58	0.19	True
LastIndexOfValue - Duration of single invocation	23.45 ns	85.33 ns	3.64	0.05	True

Uh oh!

[Perf] Linux/arm64: 6 Regressions on 4/12/2023 8:39:35 PM #16557

Description

Run Information

Regressions in System.Collections.Tests.Perf_BitArray

Repro

Payloads

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Payloads

Histogram

System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 4)

Description of detection logic

System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512)

Description of detection logic

System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 512)

Description of detection logic

Docs

Run Information

Regressions in System.Memory.Span<Int32>

Repro

Payloads

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Payloads

Histogram

System.Memory.Span<Int32>.LastIndexOfValue(Size: 512)

Description of detection logic

System.Memory.Span<Int32>.LastIndexOfValue(Size: 4)

Description of detection logic

System.Memory.Span<Int32>.LastIndexOfValue(Size: 33)

Description of detection logic

Docs

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions