feat(stats): add HyStart++ stats#3432
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3432 +/- ##
==========================================
- Coverage 94.30% 94.17% -0.13%
==========================================
Files 127 131 +4
Lines 38708 39041 +333
Branches 38708 39041 +333
==========================================
+ Hits 36502 36768 +266
- Misses 1365 1424 +59
- Partials 841 849 +8
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Merging this PR will not alter performance
Comparing Footnotes
|
- `CongestionEvent` if loss or ecn - `Heuristic` if algorithm (e.g. HyStart++)
f401181 to
1b8ad69
Compare
There was a problem hiding this comment.
Pull request overview
Adds additional congestion-control statistics to better observe HyStart++ behavior and slow-start exits, integrating them into the transport stats surface and updating CC/tests accordingly.
Changes:
- Introduces
SlowStartExitReasonand recordsslow_start_exit_reasonwhen leaving slow start via congestion events vs heuristic exit. - Adds HyStart++ CSS counters (
css_entries,css_rounds_finished) and increments them inside the HyStart++ implementation. - Updates debug formatting and HyStart++/classic CC tests to cover the new stats.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
neqo-transport/src/stats.rs |
Adds the new exit reason enum + new congestion control stats fields; updates Stats debug output. |
neqo-transport/src/cc/hystart.rs |
Increments CSS entry/round counters during HyStart++ processing. |
neqo-transport/src/cc/classic_slow_start.rs |
Updates the slow start trait impl signature to accept stats mutably. |
neqo-transport/src/cc/classic_cc.rs |
Records slow_start_exit_reason for heuristic exits and congestion events; updates tests accordingly. |
neqo-transport/src/cc/tests/hystart.rs |
Extends HyStart++ tests to validate the new stats. |
You can also share your feedback on Copilot code review. Take the survey.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Oskar Mansfeld <dev@omansfeld.net>
Client/server transfer resultsPerformance differences relative to 6a953c2. Transfer of 33554432 bytes over loopback, min. 100 runs. All unit-less numbers are in milliseconds.
Table above only shows statistically significant changes. See all results below. All resultsTransfer of 33554432 bytes over loopback, min. 100 runs. All unit-less numbers are in milliseconds.
Download data for |
Signed-off-by: Oskar Mansfeld <git@omansfeld.net>
Benchmark resultsSignificant performance differences relative to bf77765. transfer/walltime/pacing-false/same-seed: 💔 Performance has regressed by +3.2665%. time: [80.279 ms 80.340 ms 80.406 ms]
change: [+3.0475% +3.2665% +3.4320] (p = 0.00 < 0.05)
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severeAll resultstransfer/1-conn/1-100mb-resp (aka. Download)/mtu-1504: No change in performance detected. time: [202.18 ms 202.55 ms 202.94 ms]
thrpt: [492.76 MiB/s 493.70 MiB/s 494.61 MiB/s]
change:
time: [-0.6060% -0.3014% +0.0053] (p = 0.05 > 0.05)
thrpt: [-0.0053% +0.3023% +0.6097]
No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severetransfer/1-conn/10_000-parallel-1b-resp (aka. RPS)/mtu-1504: No change in performance detected. time: [284.40 ms 286.19 ms 288.01 ms]
thrpt: [34.721 Kelem/s 34.942 Kelem/s 35.162 Kelem/s]
change:
time: [-0.5411% +0.4177% +1.3542] (p = 0.39 > 0.05)
thrpt: [-1.3361% -0.4160% +0.5440]
No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mildtransfer/1-conn/1-1b-resp (aka. HPS)/mtu-1504: No change in performance detected. time: [38.552 ms 38.725 ms 38.919 ms]
thrpt: [25.694 B/s 25.823 B/s 25.939 B/s]
change:
time: [-0.4553% +0.1236% +0.7920] (p = 0.69 > 0.05)
thrpt: [-0.7858% -0.1234% +0.4573]
No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
2 (2.00%) low mild
3 (3.00%) high mild
6 (6.00%) high severetransfer/1-conn/1-100mb-req (aka. Upload)/mtu-1504: Change within noise threshold. time: [203.69 ms 204.16 ms 204.75 ms]
thrpt: [488.40 MiB/s 489.82 MiB/s 490.94 MiB/s]
change:
time: [-1.1756% -0.8715% -0.5155] (p = 0.00 < 0.05)
thrpt: [+0.5181% +0.8792% +1.1896]
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severestreams/walltime/1-streams/each-1000-bytes: Change within noise threshold. time: [590.75 µs 593.37 µs 596.28 µs]
change: [+0.4261% +1.1468% +1.8402] (p = 0.00 < 0.05)
Change within noise threshold.
Found 16 outliers among 100 measurements (16.00%)
2 (2.00%) low mild
14 (14.00%) high severestreams/walltime/1000-streams/each-1-bytes: Change within noise threshold. time: [12.430 ms 12.451 ms 12.473 ms]
change: [+0.3085% +0.7733% +1.1301] (p = 0.00 < 0.05)
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mildstreams/walltime/1000-streams/each-1000-bytes: No change in performance detected. time: [44.712 ms 44.807 ms 44.957 ms]
change: [-0.3287% -0.0829% +0.2467] (p = 0.66 > 0.05)
No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
2 (2.00%) high mild
1 (1.00%) high severetransfer/walltime/pacing-false/varying-seeds: Change within noise threshold. time: [78.912 ms 79.019 ms 79.172 ms]
change: [+1.3339% +1.4967% +1.7261] (p = 0.00 < 0.05)
Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) low mild
3 (3.00%) high mild
2 (2.00%) high severetransfer/walltime/pacing-true/varying-seeds: Change within noise threshold. time: [80.308 ms 80.406 ms 80.543 ms]
change: [+0.9533% +1.1907% +1.4058] (p = 0.00 < 0.05)
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severetransfer/walltime/pacing-false/same-seed: 💔 Performance has regressed by +3.2665%. time: [80.279 ms 80.340 ms 80.406 ms]
change: [+3.0475% +3.2665% +3.4320] (p = 0.00 < 0.05)
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severetransfer/walltime/pacing-true/same-seed: Change within noise threshold. time: [80.345 ms 80.450 ms 80.605 ms]
change: [+0.9532% +1.1326% +1.3631] (p = 0.00 < 0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severeDownload data for |
Failed Interop TestsQUIC Interop Runner, client vs. server, differences relative to
All resultsSucceeded Interop TestsQUIC Interop Runner, client vs. server neqo-pr as client
neqo-pr as server
Unsupported Interop TestsQUIC Interop Runner, client vs. server neqo-pr as client
neqo-pr as server
|
Depends on #3278. I will rebase once that lands on main. Only commits belonging to this PR are the following:add
slow_start_exit_reasonstatCongestionEventif loss or ecnHeuristicif algorithm (e.g. HyStart++)add
css_rounds_finishedstatadd
css_entriesstat to count how often css was enteredThe two CSS stats can be used to judge how often HyStart++ spuriously enters CSS and thus spends (unnecessary) time in throttled growth. In a perfect world we'd have 1 css entry and 5 rounds finished. If values are wildly increased that might point to an inefficiency.
The exit reason stat can be used to judge how well HyStart++ (or another heuristic) performs at detecting an exit point before packet loss occurs. Can also be used as a filter when recording stats on the firefox side as glean probes, e.g. to differentiate
slow_start_exit_accuracy(introduced in this bug) for heuristic and congestion event based exits.