Quality Engineering Analysis: 10-Report Assessment (Security, Performance, Testing, Architecture)

[proffesor-for-testing]

Quality Engineering Analysis - RuView (Rust Codebase)

Date: 2026-03-06
Analyzed by: AQE Queen Swarm (6 specialist agents analyzing actual source code)
Scope: Rust port (15 workspace crates + 65 WASM edge modules + ESP32 firmware)

Full 10-document report (4,836 lines, 260KB): https://gist.github.com/proffesor-for-testing/02321e3f272720aa94484fffec6ab19b

---

Overall Assessment

Dimension	Grade	Verdict
Code Quality & Architecture	B+	Excellent DDD, critical monolith in sensing-server
Security	F	7 CRITICAL findings — NOT production-ready
Performance	C+	54K fps claim unsubstantiated; no real benchmarks
Test Quality	B-	~2,195 Rust tests, 95% modules covered, critical gaps
Product Maturity	B-	48 ADRs, 8 DDD models, life-safety claims unverified
Developer Experience	A-	Comprehensive docs, Docker quickstart, clear crate graph

---

Top 10 Critical Findings

#	Finding	Severity
1	Unauthenticated OTA firmware endpoint — anyone on the network can reflash ESP32 (CVSS 10.0)	CRITICAL
2	Fake HMAC in secure_tdm.rs — XOR fold with hardcoded key, zero crypto protection	CRITICAL
3	Sensing WebSocket has zero authentication — any client gets real-time pose data (CVSS 9.1)	CRITICAL
4	WASM upload without mandatory signatures — unsigned modules loadable on ESP32 edge	CRITICAL
5	sensing-server/main.rs is 3,741 lines — CC=65, 37-field god object, untestable monolith	CRITICAL
6	54K fps claim has no supporting benchmark — no criterion benchmarks exist	CRITICAL
7	Zero security tests in Rust codebase — no auth, injection, or protocol tampering tests	CRITICAL
8	Vital sign false-negative risk — breathing/heartbeat accuracy unverified; missed survivor = missed rescue	HIGH
9	324 unsafe usages across 65 files — missing # Safety documentation	HIGH
10	O(n²) autocorrelation in heart rate detection — brute-force lag instead of FFT-based	HIGH

---

Strengths Worth Preserving

• Exceptional architecture documentation — 48 ADRs, 8 DDD domain models
• Rust error hierarchy — #[non_exhaustive], is_recoverable(), structured fields
• WASM edge module system — 65 hot-loadable sensing modules, excellent test coverage (65/66)
• Signal processing math — Kalman filter, Welford statistics, field model are mathematically sound
• Deterministic proof system — verify.py with SHA-256 hash
• 16-crate workspace — clean dependency graph with defined publishing order
• DX — 30-second Docker quickstart, comprehensive build guides

---

Recommended Priority Actions

Sprint 1: Security (Blocks Deployment)

1. Fix unauthenticated OTA endpoint (SEC-026)
2. Replace fake HMAC with real HMAC-SHA256 (SEC-004)
3. Add WebSocket authentication (SEC-002)
4. Require WASM module signatures (SEC-005)

Sprint 2: Testability & Measurement

5. Decompose main.rs into ~14 focused modules
6. Build criterion benchmarks for pipeline hot paths
7. Add security test suite
8. Fuzz CSI frame parsers with cargo-fuzz

Sprint 3: Functional Verification

9. Vital sign accuracy verification against reference signals
10. WiFi-Mat triage classification accuracy tests
11. Fix O(n²) autocorrelation
12. Multi-person tracking identity swap tests

---

Full Report Contents

#	Report	Focus
00	Executive Summary	Overall grades, top 10 findings, priority matrix
01	Code Complexity & Smells	60 findings, CC analysis, god objects, duplication
02	Security Analysis	47 findings (7 CRITICAL), OWASP, firmware, crypto
03	Performance Analysis	32 findings, O(n²) algorithms, missing benchmarks
04	Test Coverage & Quality	2,195 tests analyzed, coverage matrix, anti-patterns
05	Product Factors (SFDIPOT)	114 test ideas, 7-factor risk analysis
06	Code Quality & QX	Architecture grades, DX/UX/OpX assessment
07	Test Strategy	Test levels, tools, CI pipeline, quality gates
08	Test Plan	4 phases, 8 weeks, 47.5 days effort
09	Exploratory Charters	10 time-boxed sessions, 40 hours total

Full report: https://gist.github.com/proffesor-for-testing/02321e3f272720aa94484fffec6ab19b

[ruvnet]

this almost entirely v1. I'm focused on v2.

[ruvnet]

Thanks for this exceptional analysis — 10 reports, 4,836 lines of detailed findings with specific file/line references. This is exactly the kind of rigorous review the project needs.

Verified Findings

I've verified each critical finding against the source code:

#	Finding	Verified	Location
1	Fake HMAC (XOR fold)	YES	secure_tdm.rs:253-258 — comments explicitly say "sufficient for testing"
2	3,741-line main.rs	YES	sensing-server/src/main.rs — confirmed exact count
3	No WebSocket auth	YES	Zero auth/token checks in Rust WS code
4	No security tests	YES	No auth, injection, or tampering tests exist
5	No criterion benchmarks	YES	54K fps claim has no supporting measurement
6	Unauthenticated OTA	Needs investigation	Not found in Rust — may be ESP32 C firmware level
7	Unsigned WASM modules	Needs investigation	WASM loader review pending

Action Plan

Created ADR-050 to track the response, organized into 3 sprints:

Sprint 1 — Security (blocks deployment):

• Replace fake HMAC with real HMAC-SHA256 (hmac + sha2 crates already in Cargo.lock)
• Add WebSocket authentication (token-based)
• Add security test suite (auth bypass, frame injection, beacon replay)

Sprint 2 — Code Quality:

• Decompose main.rs (3,741 -> ~14 focused modules, none >500 lines)
• Add criterion benchmarks for pipeline hot paths

Sprint 3 — Functional Verification:

• Vital sign accuracy against reference signals
• Fix O(n^2) autocorrelation (after profiling confirms impact)

What I agree with strongly

• The fake HMAC is the most urgent fix — it's a zero-security placeholder marked as "testing only" that was never replaced
• main.rs at 3,741 lines with CC=65 is genuinely untestable and should have been decomposed long ago
• Life-safety claims (disaster survivor detection, vital signs) absolutely require verified accuracy before any production use — this is correctly flagged as a HIGH risk
• The strengths section is fair — the architecture documentation, error hierarchy, and WASM module system are genuine differentiators

Branch

Work is tracked on fix/quality-engineering-170. Security fixes will be prioritized first.

[ruvnet]

Security Hardening Complete — ADR-050 Sprint 1 Merged ✅

Thanks again to @proffesor-for-testing for the thorough analysis. We've addressed the critical findings in PR #172:

Fixes Shipped

Finding	Fix	Status
Fake HMAC (XOR fold in secure_tdm.rs)	Replaced with real HMAC-SHA256 via hmac + sha2 crates	✅ Fixed
Path traversal in DELETE endpoints	Added file_name() sanitization — rejects ../ in IDs	✅ Fixed
Server binds 0.0.0.0	Default changed to 127.0.0.1, configurable via --bind-addr	✅ Fixed
Unauthenticated OTA endpoint	Added Bearer PSK auth with constant-time comparison	✅ Fixed
WASM signatures off by default	Flipped to default-on (CONFIG_WASM_SKIP_SIGNATURE to opt out)	✅ Fixed
Zero security tests	Added 6 HMAC security tests (key sensitivity, bit-flip, wrong-key, enforcing mode)	✅ Fixed

Test Results

• 🟢 106/106 Rust hardware crate tests pass (including 6 new security tests)
• 🟢 Sensing server compiles clean
• 🟢 45/45 Python sensing tests pass

Remaining (Sprint 2-3)

• main.rs decomposition (3,741 lines → ~14 modules)
• Criterion benchmarks for performance claims
• Full WebSocket auth middleware
• Vital sign accuracy verification
• O(n²) autocorrelation optimization (if profiling confirms)

ADR-050 documents the full plan: docs/adr/ADR-050-quality-engineering-security-hardening.md