Performance benchmarking suite for different GPU types

[Defilan]

Description

Create comprehensive performance benchmarking suite to validate and compare GPU performance across different hardware.

Goals

• Automated benchmarking script
• Standard test models (3B, 7B, 13B)
• Metrics collection (tok/s, latency, memory usage)
• Comparison reports

GPU Types to Benchmark

• NVIDIA L4 (baseline complete in v0.2.0)
• NVIDIA T4
• NVIDIA A100
• AMD MI250 (when ROCm support added)
• Intel Data Center GPU Max (when oneAPI support added)

Metrics to Collect

• Prompt processing tokens/sec
• Generation tokens/sec
• P50/P95/P99 latency
• GPU memory usage
• Power consumption
• Cost per 1K tokens
• Model loading time

Success Criteria

• Benchmark script in scripts/benchmark/
• Results stored in structured format (JSON/CSV)
• Markdown report generator
• CI integration for regression testing
• Documentation in docs/benchmarks/

Sprint

Sprint 2-3 priority

Current Baseline

Performance data from Sprint 0 available in docs/gpu-performance-sprint0.md