ASR (Automatic Speech Recognition)
"ASR is the speech-to-text layer of a voice agent that turns a caller's spoken audio into text in real time so the language model can act on it."
What is an ASR (Automatic Speech Recognition)?
ASR is the speech-to-text layer of a voice agent that turns a caller's spoken audio into text in real time so the language model can act on it. Related product: Voice Agents.
Reviewed by Jaffar Jawed, Co-Founder · Last reviewed
Deep Dive
Every voice agent starts with listening. ASR is the layer that takes the audio arriving from the phone line and produces the words the rest of the system reasons over. If it hears "42 Elm Street" as "forty-two Elm's treat", nothing downstream can recover, so the quality of this one component sets the ceiling for the whole call.
How it works. Audio from the telephony leg arrives as a stream of small chunks, usually 20-100 milliseconds each, after noise suppression and echo cancellation. The recognizer emits partial transcripts while the caller is still speaking and a final transcript when end-of-speech detection decides the caller has finished. The output is text plus timing and a confidence score per segment. The agent can start reasoning on partial text, which is what lets it reply inside the pause a person would leave. Batch ASR, which waits for a complete recording, is used for post-call transcripts rather than live turns.
Why it matters for voice agents. Recognition errors turn into wrong answers, wrong bookings and wasted transfers. A misheard phone number means the confirmation text goes to a stranger. A misheard product name sends the retrieval step to the wrong document. Slow end-of-speech detection makes the agent either interrupt the caller or leave an awkward gap. Accuracy on phone audio, which is band-limited and compressed on many carrier routes, is the number that matters, not accuracy on studio recordings.
Example. A caller rings a plumbing company at 7 pm: "my water heater's leaking, I'm at 1180 Maple Court, unit 4B." Streaming ASR delivers the text with a high-confidence segment for the street and a low-confidence segment for the unit. The agent asks the caller to confirm the unit number, captures "water heater leak" as the issue, checks the service area and offers the next slot. Without the confidence signal the agent would have booked a visit to the wrong door.
ASR vs TTS. ASR converts speech to text on the way in; text-to-speech converts the reply to audio on the way out. They are separate models with separate latency budgets, and a weakness in one is not fixed by strength in the other. Natural language understanding sits between them and turns the transcript into intent and fields.
What to measure.
| Metric | What it tells you |
|---|---|
| Word error rate (WER) | Share of words substituted, dropped or inserted against a human transcript |
| Entity accuracy | Whether names, numbers and addresses survive intact |
| End-of-speech latency | Time from the caller's last word to the final transcript |
| Confidence calibration | Whether low scores really mark the errors |
Test on your own recorded calls, including mobile callers, accents and the vocabulary of your trade, and repeat the test after every configuration change.
Botcadence voice agents target sub-second response times with natural barge-in, which depends on streaming recognition; measured latency varies with model, telephony path and network conditions. See voice agents and the notes on telephony noise and echo cancellation.
Related Technology
See ASR (Automatic Speech Recognition) in a Botcadence agent
Start on the Free plan and configure an agent that uses ASR (Automatic Speech Recognition) on your own website chat or WhatsApp number. No card needed.
Get Started FreeWebsite chat live in about 5 minutes
Common Questions
ASR (Automatic Speech Recognition) FAQs
Usually yes. Mobile codecs compress audio to a narrow band, and speakerphone adds room echo and background noise. Test with recordings made on real phones rather than laptop microphones, and enable noise suppression on the telephony leg.
It cannot see the error directly, so the design relies on confidence scores and read-back. Low-confidence names, numbers and addresses are repeated to the caller for confirmation before anything is booked or written to the CRM.
Most engines accept a custom vocabulary or keyword boost list, so product names, drug names or part numbers are recognized more often. Supply the list at setup and refresh it whenever your catalog changes.
Yes. Streaming ASR produces partial text while the caller is still talking so the agent can reply quickly. Post-call transcription runs on the full recording and is usually more accurate because it has the whole sentence for context.