Skip to content
Voice AI

AI Voice Agent Latency: What Is Acceptable and Where the Milliseconds Go

How fast an AI voice agent needs to respond and why: what research on human conversation and the ITU's telephone guidance say, a budget for each stage of a call from Twilio's published targets, what vendors report, and how to measure latency on your own calls instead of trusting a headline number.

Jaffar Jawed · Co-Founder
Published
8 min read
  • Latency
  • Voice AI
  • Call Quality
  • Technical Guide
In short

Quick answer

For an AI voice agent, about one second from the end of the caller's speech to the start of the reply is a workable target: Twilio's published guidance sets 1,115 ms as the target and 1,400 ms as the limit, while people answer each other in about 200 ms. Vendor figures usually cover one stage only. Measure it on recorded calls to your own AI voice agent.

What latency is acceptable for an AI voice agent?

People leave gaps of roughly 200 milliseconds between turns in conversation, and no production voice agent on a phone line matches that today. A useful working target comes from Twilio's published guidance: about 1.1 seconds from the end of the caller's speech to the start of the agent's reply, with 1.4 seconds as an upper limit. Under about a second feels responsive, between one and two seconds feels slow but usable, and beyond two seconds callers start talking over the agent or hang up. Those bands are our reading of the sources below, not a standard. Botcadence's AI voice agent publishes no latency figure, because the result depends on the models, the carrier and the network on each call.

Key takeaways

  • Research across ten languages found the most common gap between turns is near zero to 200 milliseconds (Stivers and colleagues, 2009).
  • ITU-T G.114 advises that one-way transmission delay under 150 ms is unnoticed by most applications and that 400 ms should not be exceeded in network planning. That is network delay only, before the agent thinks.
  • Twilio's guide sets targets of 350 ms for speech recognition, 375 ms for the language model's first token and 100 ms for the voice's first audio, about 1,115 ms mouth to ear.
  • Vendor figures usually describe one stage, not the whole turn. ElevenLabs lists about 100 ms for one voice model, "excluding application and network latency".
  • Measure it yourself: record test calls and time the gap.

What the research says about human turn-taking

A study of ten languages published in PNAS in 2009 found that speakers aim for minimal gaps: the most frequent gap in each language fell between 0 and 200 milliseconds, with an overall median of about 100. A 2015 review by Levinson and Torreira describes gaps "of the order of 200 ms" even though producing a reply takes over 600 ms, which means people start planning their answer before the other person finishes.

A voice agent cannot yet do that reliably on a phone call. It waits to be confident the caller has finished, then transcribes, thinks and speaks. That is why the best systems feel slightly slower than a person and why the waiting step matters as much as the model.

What the telephone network adds

ITU-T Recommendation G.114 covers one-way transmission time on voice networks. It says that if delay is kept below 150 ms most applications are not significantly affected, and that 400 ms should not be exceeded for general network planning. Phone audio crossing carriers and the internet to reach the agent, and the reply coming back, both spend part of the budget before any AI is involved.

Where the time goes in one turn

StageWhat happensTwilio's targetTwilio's upper limit
End of turnThe agent decides the caller has finishedPart of the speech stage
Speech recognitionAudio becomes text350 ms500 ms
Language modelThe first words of the reply are generated375 ms750 ms
VoiceThe first audio of the reply is produced100 ms250 ms
Network and telephonyAudio travels each wayThe remainder
Mouth to earEnd of caller's speech to start of reply1,115 ms1,400 ms

These are Twilio's figures from its November 2025 guide to latency in voice agents. They are a vendor's targets, not measurements of any product.

The stage people overlook is the first. To avoid cutting callers off, an agent waits for a pause before it replies. Wait too briefly and it interrupts people who are thinking; wait too long and every turn feels slow. Modern systems use a model to judge whether the caller has finished, not just a silence timer. LiveKit documents a turn detector of this kind that works on the audio in 14 languages.

What vendors publish

VendorPublished figureWhat it covers
Twilio1,115 ms target, 1,400 ms limitThe whole turn, as a target
Deepgram300 ms or less for streaming transcriptionSpeech recognition only
ElevenLabsAbout 100 ms median for its Flash v2.5 voice modelVoice only, excluding application and network latency
OpenAI Realtime"Low first-audio latency", with no numberSpeech-to-speech model
CartesiaNo number on its model pageVoice

Add the single-stage figures and you do not get the experience on a call, because the end-of-turn wait, the network, the knowledge lookup and any tool call sit between them. When a vendor quotes one number, ask which stage it measures, at what percentile, and on a phone line or in a browser.

What makes a turn slower

  • A knowledge lookup. Searching your documents before answering adds time. Botcadence allows up to 1.5 seconds for it.
  • A tool call. Checking a calendar or an order means waiting for another system.
  • A larger model. Better reasoning usually costs speed.
  • A long prompt. More text to process on every turn.
  • Distance. Servers far from the caller add network delay on both legs.
  • Noise and echo. They confuse the end-of-turn decision and cause false starts.

Good agents hide part of this. A short acknowledgment such as "let me check that" while a tool runs is honest and keeps the caller oriented. Filler that pretends to be thinking is not.

How to measure latency yourself

  1. Record ten real test calls on the phone line you will use, not in a browser.
  2. In an audio editor, mark the end of your last word and the start of the agent's first word for each turn.
  3. Note the median and the worst turn. One three-second turn is what callers remember.
  4. Repeat with a question that needs a lookup and one that needs a booking.
  5. Repeat from a mobile phone with poor signal.
  6. Ask the vendor for per-call latency data and compare it with your timings.

Our guide to how to test an AI voice agent has the rest of the checklist.

Speed against accuracy

Faster is not always better. An agent that replies instantly but interrupts, mishears an address or guesses a booking time is worse than one that takes a second and gets it right. The settings that decide this are the end-of-turn sensitivity, the model, and whether the agent reads details back. Tune them on recordings of your own callers.

Where Botcadence stands

Botcadence does not publish a response-time figure, and you should be wary of any single number from any vendor. Its voice agents use a turn-detection model with an adaptive wait, stop speaking when the caller interrupts, and let you choose among several speech, voice and language model providers, which is the main lever on speed. Each call's per-turn timings are recorded internally and are not shown in the dashboard, so for now the practical test is the one above: record calls and time them. The glossary entries on barge-in and sub-500ms latency explain the terms, and the AI receptionist for small business page shows the agent in use.

Questions readers ask

AI Voice Agent Latency FAQ

About one second from the end of the caller's speech to the start of the reply is a realistic target on a phone line. Twilio's guide sets 1,115 milliseconds as a target and 1,400 as an upper limit. Past two seconds, callers tend to talk over the agent.

People start planning a reply before the other speaker finishes and leave gaps of about 200 milliseconds. An agent has to wait until it is confident the caller is done, then transcribe, generate a reply and produce speech, with network delay on both legs.

Five stages: deciding the caller has finished, speech recognition, the language model, voice synthesis, and the network and telephony in both directions. Knowledge lookups, tool calls, larger models and long prompts each add more.

Ask what it measures. Many figures cover one stage, such as voice synthesis or transcription, and exclude the network and the end-of-turn wait. Ask for the whole-turn figure on a phone line, at the median and the slowest tenth, then time your own test calls.

Record test calls on the real phone line, open them in an audio editor and mark the end of your speech and the start of the agent's reply for each turn. Record the median and the worst case, and repeat with questions that need a lookup or a booking.

Written by

· Co-Founder, Botcadence

Jaffar Jawed co-founded Botcadence in 2025 and leads product and go-to-market. He writes about AI voice agents, inbound answering workflows, lead qualification and the telephony stack behind real-time voice, including the company's open-source work on LiveKit noise and echo cancellation.

Get Started Today

See These Strategies in Action

Book a free demo and we will build a working AI agent for the channel this article covers, tailored to your business goals. No commitment required.

  • Voice AI plan $299 a month
  • Managed voice live in about 5 business days
  • Works with your existing tools

Ask AI about this page