[Perf] Linux/arm64: 4 Regressions on 12/26/2024 9:20:31 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 22.04
Queue	AmpereUbuntu
Baseline	f3d43efa514a87f04aa994749409297e8f3e94e2
Compare	194ad1667552ed2538bbbb83e336979b02ae6482
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	4.15 μs	4.56 μs	1.10	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>*'

System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToWriter(Mode: SourceGen)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 22.04
Queue	AmpereUbuntu
Baseline	f3d43efa514a87f04aa994749409297e8f3e94e2
Compare	194ad1667552ed2538bbbb83e336979b02ae6482
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	14.58 μs	16.53 μs	1.13	0.23	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>*'

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToWriter(Mode: SourceGen)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 22.04
Queue	AmpereUbuntu
Baseline	f3d43efa514a87f04aa994749409297e8f3e94e2
Compare	194ad1667552ed2538bbbb83e336979b02ae6482
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	161.87 ns	174.63 ns	1.08	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>*'

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToWriter(Mode: SourceGen)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 22.04
Queue	AmpereUbuntu
Baseline	f3d43efa514a87f04aa994749409297e8f3e94e2
Compare	194ad1667552ed2538bbbb83e336979b02ae6482
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Basic

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
WriteBasicUtf8 - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	2.16 ms	2.62 ms	1.21	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 100000)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[DrewScoggins]

Related to #101356

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

[DrewScoggins]

Windows Regression: dotnet/perf-autofiling-issues#47522

[PranavSenthilnathan]

I was able to repro this regression and it's due to the extra validation when JsonWriterOptions.SkipValidation is false. The tests exercising writing values in a tight loop (for example a collection with many primitive elements) see the perf degradation. It's surprising because the JIT output shows that the only change is an extra comparison but in a loop this seems to add up. Note that perf is not affected when skipping validation.

@eiriktsarpalis should we be concerned about this?

[eiriktsarpalis]

should we be concerned about this?

I think we should be. The impacted benchmarks are some of the most important in STJ. Could you check if there's a way to make the new functionality pay-for-play?