Skip to content

Commit c2d5460

Browse files
committed
Update hunt to use targetted benchmarks
1 parent 7316dc0 commit c2d5460

1 file changed

Lines changed: 35 additions & 10 deletions

File tree

  • .claude/skills/lading-optimize-hunt

.claude/skills/lading-optimize-hunt/SKILL.md

Lines changed: 35 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,8 @@ Otherwise, check pending hunt issues or pick from hot subsystems:
6262
| `lading_throttle` | Capacity calculation, rate limiting |
6363
| `lading` | Generators, blackholes, target management |
6464

65+
**CRITICAL: The target must have an associated criterion benchmark.**
66+
6567
---
6668

6769
## Phase 2: Analyze Target
@@ -97,23 +99,48 @@ Otherwise, check pending hunt issues or pick from hot subsystems:
9799

98100
**CRITICAL: Capture baseline metrics BEFORE making any code changes.**
99101

102+
### Identify the Benchmark Target
103+
104+
Each `lading_payload` source module has a matching benchmark target. Use `--bench <name>` to run **only** the relevant benchmark instead of the full suite.
105+
106+
| Source module | `--bench` target | Fingerprint config dir |
107+
|---|---|---|
108+
| `apache_common.rs` | `apache_common` | `ci/fingerprints/apache_common/` |
109+
| `ascii.rs` | `ascii` | `ci/fingerprints/ascii/` |
110+
| `block.rs` | `block` | *(none — use json)* |
111+
| `datadog_logs.rs` | `datadog_logs` | `ci/fingerprints/datadog_logs/` |
112+
| `dogstatsd.rs` | `dogstatsd` | `ci/fingerprints/dogstatsd/` |
113+
| `fluent.rs` | `fluent` | `ci/fingerprints/fluent/` |
114+
| `json.rs` | `json` | `ci/fingerprints/json/` |
115+
| `opentelemetry_log.rs` | `opentelemetry_log` | `ci/fingerprints/otel_logs/` |
116+
| `opentelemetry_metric.rs` | `opentelemetry_metric` | `ci/fingerprints/otel_metrics/` |
117+
| `opentelemetry_traces.rs` | `opentelemetry_traces` | `ci/fingerprints/otel_traces/` |
118+
| `splunk_hec.rs` | `splunk_hec` | `ci/fingerprints/splunk_hec/` |
119+
| `syslog.rs` | `syslog` | `ci/fingerprints/syslog/` |
120+
| `trace_agent.rs` | `trace_agent` | `ci/fingerprints/trace_agent_v04/` |
121+
122+
Set these once and use them throughout:
123+
124+
```bash
125+
BENCH=<target> # e.g. json, syslog, dogstatsd
126+
CONFIG=ci/fingerprints/<config_dir>/lading.yaml
127+
```
128+
100129
### Stage 1: Micro-benchmark Baseline
101130

102-
Use `cargo criterion` for micro-benchmarks.
131+
Run **only** the benchmark for your target:
103132

104133
```bash
105-
cargo criterion 2>&1 | tee /tmp/criterion-baseline.log
134+
cargo criterion --bench "$BENCH" 2>&1 | tee /tmp/criterion-baseline.log
106135
```
107136

108137
**Note:** Criterion stores baseline data automatically for later comparison.
109138

110139
### Stage 2: Macro-benchmark Baseline
111140

112-
Choose a config file that exercises your target code path:
141+
Use the matching fingerprint config:
113142

114143
```bash
115-
# Common configs: ci/fingerprints/{json,syslog,dogstatsd}/lading.yaml
116-
CONFIG=ci/fingerprints/json/lading.yaml
117144
cargo build --release --bin payloadtool
118145
hyperfine --warmup 3 --runs 10 --export-json /tmp/baseline.json \
119146
"./target/release/payloadtool $CONFIG"
@@ -145,10 +172,10 @@ ci/validate
145172

146173
### Stage 1: Micro-benchmarks (inner loops)
147174

148-
Re-run the same criterion benchmarks with your changes:
175+
Re-run the **same** benchmark target with your changes:
149176

150177
```bash
151-
cargo criterion 2>&1 | tee /tmp/criterion-optimized.log
178+
cargo criterion --bench "$BENCH" 2>&1 | tee /tmp/criterion-optimized.log
152179
```
153180

154181
Note: Criterion automatically compares against the last run and reports percentage changes.
@@ -166,11 +193,9 @@ Example output looks like: `time: [1.2345 ms 1.2456 ms 1.2567 ms] change: [-5.12
166193

167194
### Stage 2: Macro-benchmarks (end-to-end payloadtool)
168195

169-
Only run this if Stage 1 showed improvement. Use the SAME config as baseline:
196+
Only run this if Stage 1 showed improvement. Use the SAME `$CONFIG` as Phase 3:
170197

171198
```bash
172-
# Use same CONFIG as Phase 3
173-
CONFIG=ci/fingerprints/json/lading.yaml
174199
cargo build --release --bin payloadtool
175200
hyperfine --warmup 3 --export-json /tmp/optimized.json \
176201
"./target/release/payloadtool $CONFIG"

0 commit comments

Comments
 (0)