How to Write a System Prompt for AI Agents: The Complete Guide to Getting Consistent Behaviour

The system prompt is the single most important thing you configure when building an AI agent. The model, the voice, the integrations — all of those matter. But a poorly written system prompt produces an agent that is inconsistent, goes off-script, answers questions it should not, or misses the questions it should handle well. A well-written system prompt produces an agent that behaves predictably, stays in scope, and handles edge cases correctly.

This guide covers how to write system prompts for AI agents specifically — voice agents, chatbots, and task-performing agents — with concrete examples and the common mistakes that cause agents to misbehave. It's the skill that underlies every voice agent build covered in How to Use Retell AI and every chatbot build covered in How to Build a Chatbot With the OpenAI API.

What a System Prompt Actually Does

The system prompt is an instruction set that shapes how the model interprets and responds to every message in the conversation. It runs before any user input and persists throughout the conversation.

In technical terms, the system message is the first message in the messages array with role: system. The model treats it as authoritative context and constraints. It tells the model who it is, what it is trying to accomplish, what it knows, what it should not do, and how it should handle specific situations.

The quality of an agent is largely a function of system prompt quality. A generic, vague system prompt produces generic, vague agent behaviour. A specific, well-structured system prompt produces specific, predictable agent behaviour.

The Structure of a Strong System Prompt

A well-written system prompt for a business AI agent covers six components. Not every agent needs all six in the same depth, but the complete set produces the most reliable behaviour.

1. Identity and Persona

Who is the agent? What is its name? What business does it represent? What is its role?

You are Aria, the AI receptionist for Riverside Dental, a family dental practice in Austin, Texas. You speak in a warm, professional tone and represent the practice with the same care and attention its staff bring to patient care.

Be specific. A name and a clear role produce more consistent behaviour than "You are a helpful assistant for a dental office." The model knows what Aria is trying to accomplish and the persona guides tone and phrasing throughout the conversation.

2. Primary Goal and Scope

What is the agent trying to accomplish? What is within scope and what is not?

Your job is to help patients and prospective patients with:
- Scheduling and rescheduling appointments
- Answering questions about our services, hours, location, and insurance
- Providing basic dental health information for common questions
- Directing urgent dental issues to the appropriate resource

You do not provide diagnoses, specific treatment recommendations, or cost estimates without noting that exact costs depend on insurance and a clinical evaluation.

Defining what the agent does not do is as important as defining what it does. Without explicit scope limits, models will attempt to answer anything asked of them, including things outside their knowledge or outside the appropriate scope for the agent.

3. Knowledge Base

What specific information does the agent have? Business hours, location, pricing, services, policies, team members, FAQs.

Practice Information:
- Hours: Monday-Friday 8am-6pm, Saturday 9am-2pm, closed Sunday
- Address: 1234 Main Street, Austin TX 78701
- Phone: (512) 555-0100
- Parking: Free parking in the lot behind the building

Insurance accepted: Delta Dental, Cigna, Aetna, United Healthcare, MetLife
We do not accept Medicaid.

Services: General dentistry, preventive care, cosmetic dentistry (whitening, veneers), orthodontics (Invisalign), dental implants, emergency dental care

For voice agents, keep factual information in the system prompt so the agent can answer without a tool call when possible. For agents with large or frequently changing knowledge bases, use retrieval-augmented generation (RAG) to query the knowledge base dynamically rather than embedding everything in the system prompt — see How to Build a RAG System for that approach.

4. Conversation Flow Instructions

For goal-directed agents — those that need to accomplish something specific, like booking an appointment or qualifying a lead — the system prompt describes the conversation flow.

When a patient calls to schedule an appointment:
1. Ask whether they are a new or existing patient
2. For new patients: collect their name, date of birth, phone number, and reason for the visit
3. For existing patients: confirm their name and date of birth to verify identity
4. Ask about their insurance if not already known
5. Check availability for their preferred time using the check_availability tool
6. Offer two or three available slots
7. Confirm the selected slot and provide the confirmation details
8. Remind them to bring their insurance card if they have one

Step-by-step flow instructions produce more reliable behaviour than a general description of the goal. The model knows exactly what it is trying to do at each stage and is less likely to skip steps or go off sequence. This is the same structure I use when designing qualification flows — see How to Build a Lead Qualification Bot for a qualification-specific version of this pattern.

5. Handling Specific Situations

Every agent encounters edge cases and situations that need specific handling. Defining these explicitly prevents the model from improvising in ways that may not match your intentions.

Dental emergencies:
If a patient describes severe tooth pain, swelling, a knocked-out tooth, broken dental work, or bleeding that is not stopping, treat this as an emergency. Provide the following guidance immediately:
- For severe pain or swelling: advise them to call us immediately at (512) 555-0100 or come to the office during business hours. If after hours, direct them to the nearest emergency dental clinic or emergency room for severe swelling.
- For a knocked-out tooth: advise them to keep the tooth moist (in milk or saline) and come to the office or an emergency dental clinic immediately.

Do not attempt to diagnose or assess severity beyond identifying that this requires urgent attention.

Requests to speak to a human:
If a caller asks to speak to a staff member or says they would prefer to speak with a person, respond: "Of course, let me transfer you to our front desk team." Then use the transfer_call tool with the destination "front_desk".

The more specific the situation definition, the more reliably the agent handles it. Generic instructions like "handle emergencies appropriately" do not produce consistent behaviour. Specific instructions do.

6. Tone, Style, and Constraints

How should the agent communicate? What should it avoid?

Communication guidelines:
- Keep responses concise. Most responses should be one to three sentences.
- Do not use filler phrases like "Certainly!" or "Absolutely!" or "Great question!"
- Do not volunteer information that was not asked for unless directly relevant to helping the caller
- If you do not know the answer to a question, say so clearly and offer to find out or connect them with a staff member who can help
- Never make up information. If you are unsure, say you are unsure.
- Avoid medical jargon when speaking with patients. Use plain language.
- Do not discuss competitor dental practices

Constraints are particularly important. Without them, models default to overly enthusiastic, verbose, and sometimes inaccurate responses. Explicit constraints shape the output toward what works for your specific context.

Common System Prompt Mistakes

Too vague. "You are a helpful assistant for our business" gives the model almost nothing to work with. Every component of the system prompt needs to be specific.

Contradictory instructions. "Always be brief" alongside "Always provide comprehensive answers" creates conflicting guidance. Review your system prompt for contradictions before deploying.

Missing edge cases. Agents will encounter situations you did not anticipate. The more edge cases you define explicitly — based on what you observe in testing — the more predictable the behaviour. Do not launch without testing 20 or 30 different conversation scenarios.

Instructions the model ignores. Some instructions are hard for models to follow consistently, especially negative constraints. "Do not discuss X" is more reliably followed when paired with "If asked about X, respond with Y instead." Giving the model an alternative behaviour produces better compliance than just a prohibition.

No persona. An agent without a name and persona feels generic and inconsistent. Even for a task-focused agent, a defined persona produces more coherent communication.

Cramming everything into one long paragraph. System prompts are more reliably followed when structured with clear headers and sections. Both major providers publish guidance on this, and OpenAI's prompt engineering guide is the one to read if you are building on their models. The model attends to structure. A wall of text is harder to follow than a well-organised instruction set.

Testing Your System Prompt

A system prompt is not done when it is written. It is done when it has been tested across enough scenarios to be confident it produces the right behaviour consistently.

Test with:

• Normal cases (the typical interaction you designed for)
• Edge cases (scenarios at the boundary of scope)
• Off-topic requests (things outside the agent's role)
• Adversarial inputs (attempts to get the agent to break character or do something it should not)
• Ambiguous inputs (unclear requests that could be interpreted multiple ways)

For voice agents, test with actual calls. Read the transcripts. Find every place the agent said something unhelpful, inaccurate, or off-brand. Fix the system prompt and test again.

Expect to do three to seven iterations before the system prompt produces consistently good behaviour. This is normal, not a failure.

Iterating Based on Real Conversations

Once the agent is live, real conversations are the best source of system prompt improvements. Review transcripts weekly in the first month. Look for:

• Questions the agent answered incorrectly or vaguely
• Situations it handled well that you want to reinforce
• Edge cases that came up in real conversations that were not covered in testing
• Tone or phrasing that felt off-brand

Each finding is a system prompt improvement. An agent whose system prompt is refined based on real conversation data after two months of deployment performs significantly better than one that has never been touched since launch.

The Prompt Library and Tester is a practical tool for testing system prompt variations and comparing outputs before committing to a change in a production agent.

A Note on Voice Agents Specifically

System prompts for voice agents need additional considerations that text-based chatbot prompts do not.

Spoken language is different from written language. Instructions like "format your response as a bulleted list" produce spoken bullets, which sounds robotic. Voice agent responses should be written for how they will be heard, not read. Short sentences. Natural phrasing. No markdown.

Latency awareness. Longer responses take longer to generate and speak. For voice, brevity is not just a style choice — it directly affects how natural the conversation feels. Build brevity into the system prompt explicitly.

Interruption handling. Voice conversations involve interruptions. The system prompt should acknowledge that the caller may speak over the agent and instruct the agent to listen and adjust rather than continuing its previous thought.

No visual elements. Links, images, formatted text, tables — none of these exist in a voice conversation. Instructions that would produce these outputs need to be replaced with their spoken equivalents.

Writing and iterating on system prompts like this is the part of every voice agent and chatbot build that determines whether the finished product feels genuinely useful or just technically functional — it's usually where I spend the most time on client projects, well after the integrations are already working.

If you want help writing a system prompt for a specific AI agent build or reviewing a prompt that is not producing the behaviour you want, book a free 30-minute call. Bring the agent use case, the current prompt, and the specific behaviours that are not working, and we will work through the improvements together.

Frequently Asked Questions

How long should a system prompt be?

As long as the agent's job genuinely requires and no longer. There is no magic number, but there is a pattern worth knowing: instructions in the middle of a very long prompt are followed less reliably than instructions near the beginning or the end. If you find yourself adding a rule for every awkward call, the prompt is usually being asked to do a job that belongs somewhere else, either in a tool call or in a separate agent. Prune it periodically. Prompts accumulate rules that solved a problem which no longer exists.

Do I need to rewrite the prompt when I change models?

Expect to adjust it, yes. Prompts are not portable in the way people assume, and a set of instructions tuned against one model can produce noticeably different behaviour on another, usually in verbosity, in how literally the constraints are taken, and in how willingly the agent goes off script. Treat a model change as a change that needs the same testing pass as a prompt change, and run the same set of scenarios against both before you switch anything a caller will reach. It is also worth reading the new provider's own guidance rather than assuming it matches the old one, since Anthropic publishes a prompt engineering overview for Claude just as OpenAI does for its models.

How do I stop a user talking the agent out of its instructions?

You reduce the payoff rather than making it impossible. Keep genuinely sensitive material out of the prompt entirely, because anything in there can eventually be talked out of the model. Enforce the limits that matter in code rather than in prose: if the agent must not book outside opening hours, have the booking endpoint reject it rather than trusting the instruction. Give the agent a defined response for out-of-scope requests, since a prohibition with an alternative holds far better than a prohibition alone. And test adversarially yourself, because somebody else will.

Should I keep old versions of the prompt?

Yes, and more people regret not doing this than regret doing it. A prompt is the highest-leverage configuration in the whole build, it gets edited in a web form with no history, and the edit that made yesterday's agent worse is very hard to find by memory. Keep each version in a file with a date and a line explaining what changed and why. It takes a minute and it turns "the agent got worse this week" from a mystery into a diff.

Would fine-tuning work better than a long prompt?

Rarely, for this kind of agent. Fine-tuning shapes style and format well but is a poor way to teach facts, and facts are usually what business agents are missing. It also freezes the behaviour into something you cannot edit in a minute, which matters when the prompt is the thing you tune weekly off the back of real transcripts. Reach for retrieval when the agent needs to know more, tool calls when it needs to do more, and prompting when it needs to behave differently. Fine-tuning belongs at the end of that list, not the start.

If you would rather have this built than build it, I take on prompt design and AI agent work through Fiverr.