Retell AI Knowledge Base: Setup Guide

Putting everything an agent might need to know directly into the system prompt works fine right up until the amount of information outgrows what a prompt can reasonably hold, or until the prompt gets so long that the model starts losing track of the parts that matter most. Retell AI's knowledge base exists for exactly this problem: instead of cramming every FAQ, policy, and product detail into the prompt, the agent retrieves the relevant pieces on demand, mid-call, based on what the caller is actually asking about.

I set this up on any Retell build where the business has more than a handful of FAQs or policies the agent needs to get right, because the alternative, a bloated prompt trying to cover everything, tends to produce an agent that is vague about the things that matter and overconfident about the things it does not actually know. This post covers how the knowledge base actually works, what to put in it, and where hallucinations sneak in even with a knowledge base attached.

What a Voice Agent Knowledge Base Does

A knowledge base is a collection of sources, documents, URLs, or pasted text, that the agent can search during a call. Once linked to an agent, the agent automatically attempts retrieval before every response, condensing the recent conversation into a short search query, finding the most relevant chunks, and feeding them to the model as context alongside the conversation itself. You do not need to change the prompt to trigger this. It happens automatically on every turn.

Retell supports uploading documents directly (PDF, DOCX, TXT, and several other formats), importing content from a URL or an entire site with the option to auto-refresh every 24 hours, and adding custom text directly for anything that does not already exist as a document. Each workspace comes with a set number of free knowledge bases before additional ones carry a cost, worth checking current pricing on since this is exactly the kind of detail that can change.

What Information to Add

Not everything belongs in a knowledge base. It works best for information that is too extensive or too specific for the prompt to hold efficiently, but essential enough that the agent needs to get it right: detailed FAQs, policy documents, product specifications, pricing tiers, troubleshooting guides. Core identity, tone, and behavior instructions still belong in the system prompt itself, not the knowledge base, since retrieval happens per response and is meant to supply facts, not shape how the agent fundamentally behaves.

How to Structure Business FAQs

Writing content for a voice agent's knowledge base is different from writing a help center article a person will scan visually. A caller hears the answer read aloud rather than skimming it, so the answer needs to lead with the actual answer in the first sentence rather than building up to it the way a written FAQ often does. Front-loading matters far more for voice retrieval than it does for a page someone reads with their eyes.

Pricing and Policy Information

The largest source of voice agent hallucination is not usually the model itself, it is contradictory or outdated content sitting in the source material. If two documents in the knowledge base disagree about a return policy or a pricing detail, the retriever has no reliable way to pick the correct one, and the model will confidently answer from whichever chunk scores highest in the retrieval, regardless of which one is actually current. Before indexing anything, archive or update whatever is stale or contradictory, rather than assuming the retrieval system will somehow know which source to trust.

Handling Unknown Questions

Define explicitly in the prompt what the agent should do when the knowledge base genuinely does not contain an answer to what the caller is asking. Without this instruction, a model will sometimes attempt to answer from general knowledge rather than acknowledging the gap, which is exactly the kind of confident wrong answer a knowledge base is supposed to prevent. An honest "let me have someone follow up on that specific question" is a better outcome than a plausible-sounding guess.

Reducing Hallucinations

Beyond resolving content conflicts, keep the knowledge base focused rather than dumping every piece of company documentation into it regardless of relevance to what callers actually ask. A narrower, cleaner set of sources retrieves more accurately than a broad one with a lot of tangential content diluting what the retriever has to choose between. Pull your top caller questions from recent call transcripts and build the knowledge base around what people actually ask, not what internally seems important to document.

Testing Knowledge Retrieval

Use Retell's live web call feature to run through your top expected questions directly and confirm the agent retrieves and answers correctly, not just that the content technically exists somewhere in the knowledge base. Retrieval quality is not guaranteed just because the source document is uploaded, chunking and query matching can miss content that is phrased differently from how a caller naturally asks about it. If you have configured a Knowledge Base Instruction to steer how the search query gets built, test that specifically, since it directly affects what gets retrieved for ambiguous or conversational phrasing.

When a Knowledge Base Is Not Enough

For information that needs to be genuinely current at the moment of the call rather than periodically refreshed, live inventory, real-time availability, an account-specific detail, a knowledge base is the wrong tool. That belongs behind a function call, the pattern covered in Retell AI Function Calling, which queries a live system directly rather than retrieving from a static, even if auto-refreshing, indexed source. Knowledge base retrieval and function calling solve different problems and often sit side by side on the same agent, retrieval for reference information, function calls for anything that needs to be checked live.

Building a knowledge base that retrieves accurately and does not introduce contradictions or hallucinations is exactly the kind of Retell AI work I take on for clients. Book a free 30-minute call and bring your existing FAQs or documentation, and we will structure it properly for voice retrieval.

Frequently Asked Questions

What is the Retell AI knowledge base?

A collection of documents, URLs, or custom text an agent can search mid-call for relevant context, retrieved automatically before every response rather than living permanently in the prompt. It is built for information too extensive to hold efficiently in the prompt itself.

What should I put in a voice agent knowledge base?

Detailed FAQs, policies, product specifications, and pricing information that the agent needs to answer accurately. Core identity and behavior instructions belong in the system prompt instead, since the knowledge base supplies facts rather than shaping how the agent fundamentally acts.

How do I stop a Retell agent from hallucinating?

Remove contradictory or outdated content from the knowledge base before indexing it, since conflicting sources are the most common cause of confident wrong answers, and give the agent explicit instructions for what to say when the knowledge base genuinely does not cover a question.

Can Retell AI answer business-specific questions?

Yes, by uploading your documentation, FAQs, or policies to a knowledge base linked to the agent, which retrieves relevant content automatically during the call based on what the caller is asking.

Should pricing be included in the knowledge base?

Yes, for reference pricing that does not change per caller or per moment. For live, account-specific, or frequently changing pricing, a function call querying a live system is more reliable than a periodically refreshed knowledge base source.

How often should a voice-agent knowledge base be updated?

URL sources can be set to auto-refresh on a regular interval, commonly every 24 hours, keeping the content current without manual re-uploading. Uploaded documents need to be manually replaced when the underlying information changes.

If you would rather have this built than build it, I take on Retell AI voice agent work through Upwork.