Voice receptionist

A voice agent is either one speech-to-speech model, or a pipeline: speech-to-text, a text LLM, then text-to-speech. Vendors publish latency and cost from different setups, so the published numbers don't compare.An AI phone agent can be one model that listens and talks, or three steps: turn speech into text, have a text model reply, turn the reply back into speech. Vendors publish speeds and prices from different setups, so I couldn't compare them fairly from the outside.

So I built one receptionist that runs on either, switchable per call. The prompt, tools, handoffs and telemetry are shared, so a measured difference comes from the architecture.So I built one receptionist that runs either way, switchable per call. Everything else is shared, so any difference I measure comes from that one choice.

Callers reach it by phone (SIP) or in the browser (WebRTC). It books, reschedules and cancels through its own MCP server: seven tools behind scoped JWT auth, identity checks with lockout and a PII-masked audit log. It answers from a clinic knowledge base, hands billing to a separate agent over A2A, and transfers to a person when needed.Callers reach it by phone or in the browser. It books, moves and cancels appointments through a booking service I built, which checks who the caller is before touching their records. It answers questions from the clinic's information, passes billing questions to a separate AI agent, and hands the call to a person when needed.

Fifteen simulated callers rang both versions: among them Spanish and Japanese speakers, an elderly caller without a booking code, one attempting a prompt injection and one with chest pain. Each call was graded four ways: the booking left in the database after hang-up, the tool calls, an LLM judge, and handoffs.Fifteen simulated callers rang both versions, including Spanish and Japanese speakers, an elderly caller without a booking code, someone trying to trick it, and a caller with chest pain. Each call was graded four ways, including checking the clinic's database after the call ended.

For speed, a bot caller timed what it actually heard over WebRTC. I checked that harness first against a fake agent that replies after exactly 700 ms: it measured 862 ms, so transport adds about 160 ms. That is reported, not subtracted, because callers hear it too.For speed, a robot caller timed what it actually heard over a real browser connection. I tested the timer first on a fake agent that always waits exactly 0.7 seconds; it measured 0.86, so the connection itself adds about 0.16 seconds. I report that rather than subtract it, because callers hear it too.

Speech-to-speech started talking about twice as fast, often with a spoken “let me check” while it ran a tool. The pipeline stayed silent until it had the answer.The single-model version started talking about twice as fast, often with a quick “let me check” while it looked something up. The three-step version stayed silent until it had the answer.

The pipeline left every booking right. Speech-to-speech once booked a Spanish-speaking caller a second appointment instead of moving the first. The LLM judge gave that call 5 out of 5; the database check caught it, which is why a pass never rests on the judge alone.The three-step version got every booking right. The single-model version once booked a Spanish-speaking caller a second appointment instead of moving the first. The AI grader gave that call full marks; only the database check caught it, which is why passing never rests on the grader alone.

Both stopped for every barge-in in about a second, and neither stopped for an “mm-hmm”. On cost, a modelled 3-minute call is about $0.06 on both the pipeline and the gpt-realtime-2.1-mini I timed; the larger gpt-realtime-2 is $0.188, or $0.562 without prompt-cache hits.Both stopped talking when a caller cut in, and neither stopped for a simple “mm-hmm”. On cost, a modelled 3-minute call came to about 6 cents on both versions I timed. The larger single model costs 19 cents, or 56 cents if its caching doesn't work.Modelled with prices as of 2026-10-21: 3 minutes, 8 caller turns, 3 tool calls. Pipeline $0.061, gpt-realtime-2.1-mini $0.060 with prompt-cache hits. docs/comparison.md