Sub-500ms Latency

Human conversation runs on gaps of a few hundred milliseconds. When a voice agent takes longer than that to start speaking, callers talk over it, repeat themselves or assume the line has dropped. The half-second figure is a target for the whole round trip, from the end of the caller's speech to the first audio of the reply, not for any one component.

How it works. The round trip has five stages, each with its own budget: end-of-speech detection (deciding the caller has finished), final transcription of the last chunk, language model time to first token, time to first audio from synthesis, and transport across the telephony route. Streaming at every stage is what makes the target reachable: the model starts on partial text, synthesis starts on the first clause, and audio starts playing before the sentence is complete. A filler such as "let me check that" can cover a slow tool call, but it is a patch rather than a fix.

Why it matters for calls. Latency shapes how the agent is perceived more than any other single property. A slow agent gets interrupted, and every interruption triggers barge-in handling, which adds more delay. Callers abandon, transfer rates go up and bookings go down. Latency also varies by time of day and by model load, so a demo that feels quick at noon may not at 6 pm.

Example. A restoration company's after-hours line rings at 11 pm. The caller says "there's water coming through the ceiling." The agent's end-of-speech detector fires, the model starts drafting on the partial transcript, and the first words, "Is anyone in danger right now?", play within the pause the caller would expect from a person. Because the reply arrived quickly, the caller stays engaged through the address capture and the on-call page. The same exchange with a two-second gap usually produces "hello? are you there?" and a restart of the whole turn.

Sub-500ms latency vs barge-in. Latency is how fast the agent starts talking; barge-in is how fast it stops when the caller starts. They are measured separately and fail separately. A fast agent that cannot be interrupted is as frustrating as a slow one.

What to measure.

Botcadence voice agents target sub-second response times with natural barge-in; measured latency depends on the model, the telephony path and network conditions, so test on the live demo line rather than a web widget. See voice agents and the entry on barge-in.