RAG (Retrieval-Augmented Generation)

Retrieval-augmented generation is the layer that connects a language model to content it was never trained on: your price list, your cancellation policy, your service area. In a chat or voice agent it sits between the customer's message and the model's reply. The message arrives, the retriever looks up the passages most likely to answer it, and the model writes from those passages instead of from general knowledge.

How it works. Ingestion splits each document into passages of a few hundred words, converts each into an embedding vector and stores it in a vector index. At query time the customer's message is embedded the same way, the index returns the closest passages (usually three to eight), and those are placed in the prompt with an instruction to answer only from them. The model's output is the reply plus, on many platforms, the passage IDs it drew on, which is what lets a reviewer trace a claim to its source. Updating a document re-indexes its passages, so the agent's answers change without touching the model.

Why it matters for support and sales teams. The business value is control. Without retrieval, an agent asked about a refund window produces a plausible number from training data. With retrieval, it produces your number, and when no passage covers the question it can say so and hand off. That turns the agent from a fluent guesser into something a support lead can audit: every answer has a source, and a wrong answer points at a document to fix rather than a model to retrain.

Example. A heating and cooling contractor uploads its service agreement, pricing sheet and a page on financing. A homeowner asks on web chat, 'does the maintenance plan cover the capacitor if it fails in summer'. The retriever returns the plan's parts-coverage clause and the exclusions paragraph. The model answers that capacitors are covered under the plan with no call-out fee, quotes the exclusion for units over 15 years old, and offers to book a technician. When the contractor changes the exclusion to 12 years the next month, the next customer gets the new answer within minutes of the upload.

RAG vs fine-tuning. Fine-tuning changes the model's weights using example conversations; it teaches style and format, and it bakes in whatever facts were in the examples at training time. Retrieval leaves the model alone and changes what it reads on each turn. For facts that change, which is nearly every business fact, retrieval is the only practical option: you cannot fine-tune every time a price moves.

Common pitfalls.

• Chunks that split a table from its header, so the price row is retrieved without the product name.
• Two documents that contradict each other, with no effective date to break the tie.
• Retrieving too many passages, which pushes the model toward a vague summary instead of a direct answer.
• No decline path: the agent should say it cannot find that information rather than fill the gap.

Botcadence chat agents answer from uploaded documents and website crawls on every plan, with the website widget typically live under 5 minutes after upload; see document training.