The voice AI eval loop for coding agents.
Real calls, in-call measurement, autonomous iteration.
See our launch on X!
Vent is a CLI for coding agents that lets them place calls and evaluate voice agents through vocal agent-to-agent communication. It works with Vapi, Retell, LiveKit, ElevenLabs, Bland, and custom endpoints.
When your coding agent runs vent-hq run, Vent places a real call to your voice agent, measures the entire call from inside, and returns a structured result. Most voice eval tools probe from outside; Vent runs inside. Think of it as a CLI version of Hamming, Roark, or Coval.
Under the hood, a Claude Haiku 4.5 caller LLM drives the conversation, speaking via Deepgram Aura-2 over WebSocket or WebRTC. Vent listens through a vendored TEN VAD WASM module (no external service call) and Deepgram nova-3 STT across seven languages. Everything that happened inside the call comes back — transcript, audio quality, latency at every layer (percentiles + per-turn TTS/STT/LLM breakdown), every tool and MCP call, every transfer, pathway and flow decisions, agent state transitions, provider warnings, cost and usage, debug URLs, and the recording — as structured JSON in .vent/runs/ with the audio at S3/R2 as a signed URL.
npx vent-hq@latest init # Agent auto-setup: auth, skills, configThe init command lets the agent automatically log in via GitHub, generate an access token, install skill files, and scaffold a starter config at .vent/suite.json. For access without GitHub, run login and then init.
npx vent-hq@latest init # Auto-setup
npx vent-hq run -f .vent/suite.json # Run all calls, stream SSE
npx vent-hq run -f .vent/suite.json --call foo # Run a specific call
npx vent-hq run -f .vent/suite.json -v # Verbose result JSON
npx vent-hq stop <run-id> # Cancel a queued or active runA minimal .vent/suite.json:
{
"connection": {
"adapter": "vapi",
"platform_connection_id": "..."
},
"calls": {
"refund_request": {
"caller_prompt": "Ask for a refund on order #1234. Get frustrated if pushed back.",
"persona": { "pace": "fast", "clarity": "clear" },
"max_turns": 8,
"language": "en",
"voice": "female"
}
}
}Credentials are encrypted at rest with AES-256-GCM and never appear in chat or run payloads.
Hosted only. Set your API key and agent/assistant ID, save as a platform connection, reference by platform_connection_id.
Test local agents in dev mode or deployed agents on LiveKit Cloud — same config, different LIVEKIT_URL. Target a specific agent with livekit_agent_name.
Since LiveKit does not forward events natively, install our first-class instrumentation libraries — @vent-hq/livekit (Node) or vent-livekit (Python).
One line:
const vent = instrumentLiveKitAgent({ ctx, session });vent = instrument_livekit_agent(ctx=ctx, session=session)Both packages hook the LiveKit Agents AgentSession lifecycle (metrics_collected, function_tools_executed, conversation_item_added, user_input_transcribed, session_usage_updated, close) and publish to 10 dedicated Vent DataChannel topics. Session report auto-publishes on session.close — the last safe window before room.disconnect. Without the instrumentation, Vent still captures agent state transitions and transcripts directly from the Agents SDK.
Test pathways (bland_pathway_id), personas (persona_id), or inline prompts (task).
Point Vent at a hosted endpoint with agent_url, or test a pure-localhost agent with start_command + agent_port — Vent spawns your agent and tunnels audio through a relay so you don't need a public IP. Your agent emits tool_call, vent:timing, vent:call-metadata, vent:transcript, vent:transfer, vent:debug-url, vent:warning, speech-update, and end-call for full observability.
For parallel calls, share one relay session — each call gets its own multiplexed connection through the relay, so the agent isn't respawned for every run:
npx vent-hq agent start -f .vent/suite.json
npx vent-hq run -f .vent/suite.json --session <session-id>
npx vent-hq agent stop <session-id>