GH-560: Wire wgpu fallback into batch inference path

[noahgift]

Five-Whys

1. Why doesn't batch mode use wgpu? → init_batch_model() in batch.rs only tries OwnedQuantizedModelCuda. When CUDA parity fails, it falls to CPU.
2. Why no wgpu in batch init? → The BatchModel struct only has gpu: Option<OwnedQuantizedModelCuda> and cpu: Option<OwnedQuantizedModel>. No wgpu variant.
3. Why no wgpu variant? → Batch was designed before wgpu inference existed.
4. Why does this matter? → 32B model in worker mode times out (316s per problem). Batch mode loads model once. Without wgpu batch, 32B eval is CPU-only (slow) or worker-mode (timeouts).
5. Fix: Add wgpu to init_batch_model — when CUDA parity fails, try wgpu before CPU.

Contract

gpu-multi-backend-parity-v1.yaml equation backend_priority:

select(backends) = first(b in [cuda, wgpu, cpu] where parity(b) >= 0.98)

Currently violated in batch path — batch skips wgpu entirely.

Acceptance Criteria

• apr run model.apr --batch-jsonl prompts.jsonl --gpu uses wgpu when CUDA parity fails
• Batch output shows "used_gpu": true with wgpu backend
• 32B MBPP eval completes without timeouts

[noahgift]

GH-560 Update: wgpu batch works but LM head bottleneck

wgpu batch is functional:

• "used_gpu": true, CUDA→wgpu fallback activates correctly
• Model dequantization: 18s (one-time)
• Per-token inference through 28 wgpu layers: fast

Bottleneck: LM head weight (152064×3584×4 = 2.2 GB) exceeds wgpu 2GB upload limit ([PMAT-377] Skipping GPU upload for lm_head). Falls back to CPU LM head matmul which takes ~50s per token (152K × 3584 dot products on aarch64 NEON).

Fix needed: Either increase wgpu buffer limit or split LM head into chunks for GPU upload. Filed separately.

[noahgift]

GH-560 Update: Two Remaining Issues

1. OOM: 28 GB bulk F32 allocation

dequant_model_weights() allocates Vec of ALL 337 weights (28 GB). Even with into_iter(), the Vec's backing memory stays allocated until fully consumed. Per-tensor streaming dequant avoids this but...

2. Fused QKV not handled

Per-tensor upload misses fused QKV weights (OwnedQKVWeights::Fused). The Q/K/V projections silently become zero → garbage output.

Root cause

Both issues stem from using F32 dequantization instead of raw Q4K upload. The Q4K GEMV wgpu shader (PMAT-363) handles both issues:

• Reads raw Q4K (7.5 GB, not 28 GB)
• Handles fused/separate QKV transparently

Fix

Update gx10 trueno to include upload_q4k_weight and init_kv_cache. Requires pushing trueno (blocked by pre-push hook format/clippy issues).

Workaround

CPU batch works reliably (84.76% HumanEval, ~3 hours).
wgpu single-prompt works (cosine=0.999863).

[noahgift]

Triage: P2 — Feature gap. Batch inference path lacks wgpu fallback (currently CUDA or CPU only). Not hardware-dependent — can be implemented on any dev machine with wgpu support.