LLM (Large Language Model)

A large language model is the component of a voice or chat agent that reads and writes. Everything else in the stack feeds it or acts on its output: speech-to-text turns audio into words the model can read, retrieval supplies passages from your documents, tools expose your calendar and CRM, and text-to-speech reads the reply back to a caller. The model itself holds no knowledge of your business beyond what sits in its prompt.

How it works. The model receives a single block of text: the system prompt with its role and rules, the retrieved passages, the conversation so far and the latest customer message. It predicts the reply one token at a time, where a token is roughly three quarters of a word. When the prompt lists tools, the model can emit a structured call such as check_availability(date) instead of prose; the platform runs the tool, appends the result and asks the model to continue. Each turn ends in either a customer-facing message or a tool call, which keeps the loop easy to log and audit.

Why it matters for answer quality and cost. Model choice sets the ceiling on comprehension and the floor on latency and price. A larger model follows long rule sets and handles messy phrasing better; a smaller model answers faster, which matters on the phone where a two-second pause reads as a dropped call. Every reply also costs tokens in and out, so a bloated system prompt is paid for on every turn. Operations leads should know which model their agent runs on, what the fallback is when that provider is slow, and whether the model can be swapped without rebuilding the agent.

Example. A Shopify store runs a chat agent for order questions. A customer writes 'where's my stuff, ordered last tuesday'. The model reads the system prompt (be brief, never guess shipping dates), notes that no order number was given, and asks for the email on the order. The customer replies, the model emits a lookup tool call, the platform returns the shipment status, and the model writes two sentences with the carrier and the tracking link. The model did not know the order existed until the tool told it.

LLM vs RAG. RAG is a pattern around the model, not a type of model. The model reasons and writes; retrieval decides which of your documents it gets to read before writing. Replacing the model changes tone, speed and reasoning; replacing the retrieval setup changes which facts are available. Most wrong answers in production trace to retrieval gaps or prompt gaps, not to the model.

Common pitfalls.

• Letting the model answer prices, hours or policies from memory instead of from a retrieved passage.
• Prompts that grow past what the model can hold comfortably, so early rules get ignored.
• No timeout or fallback when the provider is slow, which shows up as dead air on calls.
• Treating a demo on one model as proof the agent works on a cheaper one.

Botcadence agents train on your uploaded documents and website in one primary language and reply in 50+ languages on chat, with the model wrapped in retrieval, tools and handoff rules; see chat agents and document training.