Twilio Voice Setup for AI Agents: How to Connect a Phone Number to Your Voice Automation Stack

Twilio is the telephony backbone behind most production AI voice agent deployments. Whether you are building on Retell AI, Vapi, or a custom voice stack, Twilio handles the phone number provisioning, the inbound call routing, and the outbound call initiation that connects your AI agent to the real phone network.

This guide covers the complete Twilio voice setup for AI agents: account configuration, number provisioning, inbound call routing to your agent, outbound calling, and the operational details that matter when calls are going to real people. If you're building the agent itself, How to Use Retell AI covers that side of the stack.

Why Twilio

Twilio is not the only telephony option for AI voice agents, but it is the most widely supported. Retell AI, Vapi, and Bland AI all have native Twilio integration. The Twilio API is mature, well-documented, and has SDKs for every major language; the Programmable Voice documentation is where the call, stream and recording behaviour used throughout this guide is defined. Support for PSTN calls (regular phone calls), SIP trunking, and programmable voice is comprehensive.

The main alternative worth knowing is a direct SIP trunk from a VoIP provider. This can be cheaper at very high call volumes and some AI voice platforms support bringing your own SIP trunk. For most builds, Twilio is the right starting point and switching to an alternative trunk later is straightforward.

Account Setup and Verification

Create a Twilio account at twilio.com. New accounts start with a free trial that includes credit and a trial phone number. The trial number has limitations — calls can only go to verified numbers — so it is useful for initial testing but you need to upgrade before production.

Upgrade to a paid account by adding a payment method and completing identity verification. Twilio requires identity verification for regulatory compliance before you can purchase phone numbers or make outbound calls to unverified numbers.

Once upgraded, navigate to the Console. The Account SID and Auth Token on the Console dashboard are your API credentials. Keep these secure. Store them in environment variables, never in code.

Purchasing a Phone Number

Navigate to Phone Numbers > Manage > Buy a Number in the Twilio Console.

Filter by country, number type (local, toll-free, or mobile), and capabilities. For AI voice agent use, you need Voice capability at minimum. If the agent will also send SMS, add SMS capability.

For local numbers, choose a number with an area code relevant to the business location. A local number in the same area code as the business builds familiarity and answer rates are higher for outbound calls from local numbers than from toll-free or out-of-area numbers.

For businesses with national scope or high inbound volume, toll-free numbers (800, 888, 877, etc.) are appropriate. They are associated with established businesses and have no per-minute cost to the caller.

After purchase, the number appears in your Active Numbers list. This is where you configure what happens when a call comes in or when you want to make an outbound call.

Connecting Twilio to Retell AI for Inbound Calls

Retell AI has native Twilio integration. In the Retell dashboard, navigate to Phone Numbers and connect your Twilio account using your Account SID and Auth Token. Retell will list your purchased Twilio numbers and allow you to assign an agent to each one.

When you assign an agent to a Twilio number in Retell, Retell automatically configures the Twilio number's webhook settings to point to Retell's infrastructure. Inbound calls to that number are answered by Retell and routed to the assigned agent.

This is the simplest configuration: one number, one agent, Retell handles everything. For more complex routing — different agents for different times of day, routing based on the caller's number or a menu selection, call forwarding to a human under specific conditions — you configure this in Retell's agent settings or through Twilio Studio.

Manual Twilio Webhook Configuration (For Non-Retell Stacks)

If you are building a custom voice stack or using a platform that does not have native Twilio integration, you configure the webhook manually.

In the Twilio Console, navigate to your phone number's configuration. Under Voice Configuration, set the webhook URL to the endpoint that will handle the incoming call. Twilio sends an HTTP POST request to this URL when a call comes in, with details about the call (caller number, called number, call SID).

Your endpoint responds with TwiML — Twilio's XML-based instruction language — that tells Twilio what to do with the call: connect it to a stream for real-time audio processing, play a message, gather input, or forward to another number.

For an AI voice agent that processes audio in real time, the typical TwiML response initiates a WebSocket stream:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Connect>
    <Stream url="wss://your-agent-endpoint.com/stream">
      <Parameter name="call_sid" value="{{CallSid}}"/>
    </Stream>
  </Connect>
</Response>

Your WebSocket endpoint receives the raw audio from the call in real time, processes it through your STT and LLM pipeline, and sends audio responses back through the WebSocket to be played to the caller. A FastAPI backend handling this endpoint follows the same architecture covered in Python FastAPI Webhook Automation.

Outbound Calling With Twilio

For outbound AI calling campaigns, Twilio initiates the calls via its REST API. You provide the number to call, the number to call from, and a URL that returns TwiML instructions for what to do when the call is answered.

from twilio.rest import Client
import os

client = Client(os.environ["TWILIO_ACCOUNT_SID"], os.environ["TWILIO_AUTH_TOKEN"])

call = client.calls.create(
    to="+14085551234",
    from_=os.environ["TWILIO_PHONE_NUMBER"],
    url="https://your-server.com/outbound-call-handler",
    status_callback="https://your-server.com/call-status",
    status_callback_event=["initiated", "ringing", "answered", "completed"]
)

print(f"Call SID: {call.sid}")

The URL parameter points to your TwiML handler, which instructs Twilio what to do when the call is answered. The status callback fires at each call lifecycle event, useful for logging and triggering post-call automation.

For Retell AI outbound campaigns, you use Retell's batch call API instead of calling Twilio directly. Retell handles the Twilio call initiation internally and manages the campaign queue, retry logic, and call pacing — see Outbound AI Calling for how a full campaign is structured on top of this.

Call Recording and Transcription

Twilio supports native call recording. Enable it on a per-call basis by adding a record="true" attribute to your TwiML verb, or enable it for all calls on a number through the Console.

Recordings are stored in Twilio and accessible via the API or the Console. For AI agent deployments, most platforms (Retell, Vapi) handle their own transcription. If you need Twilio's native transcription, add transcribeCallback to capture the transcript at call completion.

For production deployments, decide upfront on your recording retention policy, and if any of your callers are in the UK work out which of the options in the ICO's guide to lawful basis you are relying on to keep the recording at all. Twilio stores recordings indefinitely by default. For HIPAA-compliant deployments or businesses with data minimisation requirements, configure automatic deletion after a defined retention period.

Caller ID and CNAM

By default, outbound calls from your Twilio number display the number with no name. Most mobile phones show "Unknown Caller" or just the number. This reduces answer rates significantly for outbound campaigns.

Twilio supports CNAM (Caller Name) registration for US numbers on some plans. Registering a name with the CNAM database means your business name appears on the caller ID for many recipients. This is not universal — carrier support varies — but it improves answer rates meaningfully.

Alternatively, use a local presence number in the same area code as the person being called. Local numbers answer at higher rates than out-of-area or toll-free numbers for outbound campaigns.

A2P 10DLC for SMS (If You Are Also Sending Texts)

If your AI agent also sends SMS — confirmation texts, follow-up messages, appointment reminders — and you are using Twilio numbers in the US, A2P 10DLC registration is required. Unregistered numbers have their messages filtered by carriers.

Register your brand and campaign in the Twilio Console under Messaging > Regulatory Compliance. The process takes one to three business days. Once approved, your SMS messages go through without filtering.

This is not optional for production SMS use. Build registration into your project timeline. If your SMS is running through GoHighLevel instead of raw Twilio, GoHighLevel SMS Automation covers the equivalent A2P setup on that platform.

Managing Costs

Twilio charges per minute for inbound and outbound calls (rates vary by country and number type), per phone number per month, and per SMS message. For production AI voice agent deployments, the Twilio cost adds to the Retell or other platform cost.

Monitor your Twilio usage in the Console. Set up billing alerts to notify you when monthly spend exceeds a threshold. For high-volume deployments, contact Twilio for volume pricing — custom rates are available above certain usage levels.

The AI Voice Agent Cost Simulator factors in Twilio costs alongside the AI platform cost to give you a complete per-minute cost for your deployment.

Production Checklist Before Going Live

Before directing real callers to your Twilio number:

• Upgrade from trial to paid account
• Complete identity verification
• Register A2P 10DLC if using SMS
• Test inbound calls from multiple phone types (mobile, landline, VoIP)
• Test outbound calls and confirm caller ID displays correctly
• Confirm call recording is configured per your policy
• Set billing alerts in the Console
• Verify the call status webhook is receiving events
• Test the emergency escalation path — make sure calls that should reach a human do so reliably

This checklist is the same one I run through on every voice agent build before a client's phone number goes live, whether it's an inbound receptionist or an outbound campaign. If you want help setting up Twilio for a specific AI voice agent deployment or troubleshooting a configuration that is not behaving as expected, book a free 30-minute call. Bring your current setup details and the specific issue and we will work through it together.

Frequently Asked Questions

Why are my outbound calls to another country failing?

Almost always geo permissions. Twilio blocks outbound calls to most countries by default and only enables the ones you tick in Voice Geographic Permissions, which exists to stop a compromised account running up a bill dialling expensive destinations. The failure is unhelpful when you meet it for the first time, because everything in your code is correct and the call simply does not go. Enable the countries you actually call and leave the rest off rather than opening everything, since the setting is doing a real job.

Can I move my existing business number to Twilio?

Yes, and porting is the right answer once the agent is doing more than after-hours overflow, but treat it as a scheduled piece of work rather than an afternoon's task. It takes days to weeks depending on the carrier and the country, the losing carrier will reject the request over trivial mismatches in the account name or service address, and the number is unusable for a short window during the cutover. Forward first, port later once you are certain the setup is staying.

How do I keep several clients' numbers separate?

Subaccounts. They give each client their own SID, their own credentials and their own usage line on the bill, which means you can hand over or shut down one client without touching another and you can see exactly what each one costs you. Doing it after the fact is painful because numbers have to be transferred between subaccounts and anything hard-coded against the parent credentials breaks. If you expect more than one client, set it up on the first one.

Why is Twilio asking for an address before it will sell me a number?

Because many countries require it. Local numbers outside the US frequently need a regulatory bundle with proof of identity and a local address, and some require the address to be in the same region as the number's area code. Approval is not instant, so a build that assumes you can buy a number the morning you need it will slip. Check the requirements for the country before you promise a go-live date, and start the bundle early since the review runs in parallel with everything else.

How many calls can one number handle at the same time?

Fewer than you would guess, and the limit sits on the account and the number rather than on your agent. A single number handling a burst of simultaneous inbound calls will start rejecting them, which sounds to the caller like the business is engaged. If you expect spikes, spread inbound across several numbers or ask Twilio to raise the limit before the spike rather than during it, and pace outbound campaigns deliberately instead of firing a whole list at once.

If you would rather have this built than build it, I take on voice and telephony integration work through Upwork.