BV
All articles

How to Build a Chatbot With the OpenAI API: A Developer's Practical Guide

Building a chatbot with the OpenAI API goes from basic API call to production-ready system once you understand conversation history, system prompts, tool calls, and streaming.

Muhammad Bilal
Muhammad Bilal Virk
5 min read
How to Build a Chatbot With the OpenAI API: A Developer's Practical Guide

How to Build a Chatbot With the OpenAI API: A Developer's Practical Guide

Building a chatbot with the OpenAI API is one of the most direct paths from a working idea to a deployed product. The API is well-documented, the Python and Node.js SDKs are mature, and the core pattern — maintain a conversation history, send it to the API, return the response — is something you can have working in under an hour.

What takes longer is building a chatbot that is actually useful in a production context: one that stays on topic, handles edge cases gracefully, maintains context across long conversations, integrates with your business data, and does not cost a fortune to run at scale.

This guide covers the full picture: from the basic API call to a production-ready chatbot with system prompts, conversation memory, tool calls, streaming, and cost control. For the business use case this often serves — capturing and qualifying leads — see AI Chatbot for Lead Generation.


The Core Pattern

The OpenAI Chat Completions API takes an array of messages and returns the next message in the conversation. Each message has a role (system, user, or assistant) and content (the text of the message).

The simplest working chatbot in Python:

python
from openai import OpenAI

client = OpenAI(api_key="your-api-key")

conversation = [
    {"role": "system", "content": "You are a helpful assistant for Bilal's automation agency. Answer questions about Make.com, n8n, GoHighLevel, and AI voice agents."},
]

def chat(user_message: str) -> str:
    conversation.append({"role": "user", "content": user_message})
    
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=conversation,
    )
    
    assistant_message = response.choices[0].message.content
    conversation.append({"role": "assistant", "content": assistant_message})
    
    return assistant_message

# Usage
print(chat("What is Make.com good for?"))
print(chat("How does it compare to Zapier?"))

The conversation list grows with each exchange. The API always receives the full history, which is how the model maintains context. The user asks a follow-up question about "it" and the model knows what "it" refers to because it has seen the whole conversation.


The System Prompt: Where Chatbot Quality Is Made

The system message is the instruction set that shapes everything the chatbot does. It is the most important thing you configure and the part most developers underinvest in. The same discipline applies whether you're building a text chatbot or a voice agent — see How to Write a System Prompt for AI Agents for the deeper treatment of this skill.

A weak system prompt: "You are a helpful assistant."

A strong system prompt for a business chatbot:

text
You are an AI assistant for Riverside Dental, a family dental practice in Austin, Texas.

Your job is to help website visitors with:
- Questions about our services (cleanings, fillings, cosmetic dentistry, orthodontics)
- Appointment booking (collect name, phone, preferred date/time, service needed)
- Insurance questions (we accept Delta Dental, Cigna, Aetna, and United Healthcare)
- General dental FAQs

Our hours are Monday-Friday 8am-6pm and Saturday 9am-2pm.
Our address is 1234 Main Street, Austin TX 78701.
Our phone number is (512) 555-0100.

If someone asks about a specific tooth pain or dental emergency, provide basic first aid guidance and strongly encourage them to call us immediately or visit an emergency dentist.

Do not provide diagnoses. Do not discuss competitor practices. Do not discuss pricing without noting that costs vary by insurance and treatment plan.

Keep responses concise. If the visitor wants to book an appointment, collect their information and confirm you will pass it to our team.

This system prompt defines scope, persona, knowledge boundaries, and behaviour for specific situations. The chatbot built on it behaves consistently and appropriately because the instructions are specific.


Managing Conversation History for Long Chats

The OpenAI API has a context window limit. For GPT-4o mini it is 128,000 tokens. For most chatbot conversations this is more than sufficient, but long sessions or sessions with large system prompts can approach the limit.

The naive solution — just keep appending to the conversation list — eventually hits the limit and fails. A production chatbot needs conversation management.

Sliding window. Keep only the system prompt and the last N exchanges. Simple to implement, loses older context.

python
MAX_HISTORY = 20  # Keep last 10 exchanges (20 messages)

def trim_conversation(conversation: list) -> list:
    system_messages = [m for m in conversation if m["role"] == "system"]
    non_system = [m for m in conversation if m["role"] != "system"]
    return system_messages + non_system[-MAX_HISTORY:]

Summarisation. When the conversation exceeds a threshold, send the older portion to the API with a "summarise this conversation so far" prompt and replace the old messages with the summary. Maintains context without hitting limits.

Token counting. Use the tiktoken library to count tokens before each API call and trim or summarise based on actual token count rather than message count. More precise than message-based trimming.

python
import tiktoken

def count_tokens(messages: list, model: str = "gpt-4o-mini") -> int:
    encoding = tiktoken.encoding_for_model(model)
    return sum(len(encoding.encode(m["content"])) for m in messages)

Streaming Responses

For any chatbot where the user sees the response as it is generated — which is the standard experience in web and chat interfaces — streaming is essential. Without streaming, the user stares at a blank screen until the full response is ready. With streaming, they see tokens appearing as the model generates them.

python
def chat_streaming(user_message: str):
    conversation.append({"role": "user", "content": user_message})
    
    stream = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=conversation,
        stream=True,
    )
    
    full_response = ""
    for chunk in stream:
        if chunk.choices[0].delta.content:
            token = chunk.choices[0].delta.content
            full_response += token
            print(token, end="", flush=True)  # Or send to your frontend
    
    conversation.append({"role": "assistant", "content": full_response})
    return full_response

For web applications, server-sent events (SSE) from a FastAPI endpoint deliver the stream to the browser in real time. The frontend displays each token as it arrives. Python FastAPI Webhook Automation covers the broader FastAPI patterns this deployment builds on.


Tool Calls: Giving the Chatbot Actions

A chatbot that can only converse is limited. A chatbot that can take actions — look up data, create records, send messages, check availability — is genuinely useful for business automation.

The OpenAI tool calling (function calling) system lets you define functions that the model can choose to invoke. When the model decides a tool call is appropriate, it returns a structured tool call object instead of a text response. Your code executes the function and passes the result back to the model.

python
tools = [
    {
        "type": "function",
        "function": {
            "name": "check_appointment_availability",
            "description": "Check available appointment slots for a given date",
            "parameters": {
                "type": "object",
                "properties": {
                    "date": {
                        "type": "string",
                        "description": "The date to check availability for, in YYYY-MM-DD format"
                    },
                    "service_type": {
                        "type": "string",
                        "description": "The type of appointment requested"
                    }
                },
                "required": ["date"]
            }
        }
    }
]

def handle_tool_call(tool_name: str, tool_args: dict) -> str:
    if tool_name == "check_appointment_availability":
        # Your actual availability check logic here
        date = tool_args.get("date")
        return f"Available slots on {date}: 9:00am, 11:00am, 2:00pm, 4:00pm"
    return "Tool not found"

def chat_with_tools(user_message: str) -> str:
    conversation.append({"role": "user", "content": user_message})
    
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=conversation,
        tools=tools,
    )
    
    message = response.choices[0].message
    
    # Check if the model wants to call a tool
    if message.tool_calls:
        conversation.append(message)  # Add the tool call message
        
        for tool_call in message.tool_calls:
            import json
            result = handle_tool_call(
                tool_call.function.name,
                json.loads(tool_call.function.arguments)
            )
            conversation.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": result
            })
        
        # Get the final response with tool results included
        final_response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=conversation,
        )
        assistant_content = final_response.choices[0].message.content
        conversation.append({"role": "assistant", "content": assistant_content})
        return assistant_content
    
    conversation.append({"role": "assistant", "content": message.content})
    return message.content

Cost Control

At scale, OpenAI API costs add up. Three practices that keep costs manageable.

Use the right model. GPT-4o mini costs roughly 15 times less than GPT-4o and handles the majority of chatbot use cases well. Use GPT-4o for complex reasoning tasks. Use GPT-4o mini for everything else. The quality difference for typical customer-facing chatbot conversations is minimal.

Cache frequent responses. If your chatbot answers the same questions repeatedly — business hours, pricing, return policy — cache the responses in Redis or a simple dictionary. Return the cached response for matching inputs without an API call.

Monitor usage. The OpenAI dashboard shows usage by day, by model, and by organisation. Set usage limits and alerts. A runaway automation scenario or an unexpected traffic spike should not result in a surprise bill.

The OpenAI Cost Calculator models your expected monthly API cost at different usage volumes and model combinations before you commit to an architecture.


Deploying as a FastAPI Endpoint

Wrapping the chatbot in a FastAPI endpoint makes it accessible to any frontend or integration.

python
from fastapi import FastAPI
from pydantic import BaseModel
from fastapi.responses import StreamingResponse
import json

app = FastAPI()

class ChatRequest(BaseModel):
    message: str
    session_id: str

sessions = {}  # In production, use Redis

@app.post("/chat")
async def chat_endpoint(request: ChatRequest):
    if request.session_id not in sessions:
        sessions[request.session_id] = [
            {"role": "system", "content": "Your system prompt here"}
        ]
    
    conversation = sessions[request.session_id]
    conversation.append({"role": "user", "content": request.message})
    
    async def generate():
        stream = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=conversation,
            stream=True,
        )
        full_response = ""
        for chunk in stream:
            if chunk.choices[0].delta.content:
                token = chunk.choices[0].delta.content
                full_response += token
                yield f"data: {json.dumps({'token': token})}\n\n"
        conversation.append({"role": "assistant", "content": full_response})
        yield "data: [DONE]\n\n"
    
    return StreamingResponse(generate(), media_type="text/event-stream")

This endpoint maintains session state per session ID, streams the response, and updates the conversation history. In production, replace the in-memory sessions dictionary with Redis for persistence across server restarts and multi-instance deployments.


What to Build Next

Once the core chatbot is working, the most valuable additions are: a knowledge base using retrieval-augmented generation (RAG) so the chatbot can answer questions from your specific documents — see How to Build a RAG System for that build — guardrails that detect and handle off-topic or inappropriate requests, and analytics that track which questions are asked most frequently so you can improve the system prompt over time.

If you want help designing a production chatbot for a specific business use case or need a review of a chatbot architecture you are building, book a free 30-minute call. Bring the use case and the current architecture and we will work through the design together.

Muhammad Bilal
Muhammad Bilal Virk
AI automation engineer — building agents, workflows, and RPA that remove repetitive work.
Share
Newsletter

One email, when I ship something worth reading.

No cadence, no filler. Unsubscribe any time.

Free consultation

Want this built against your real numbers?

A 30-minute call to scope the workflow, agent, or automation you actually need.

Book a free consultation
Next step

Have a workflow that's burning hours every week?

Bring me one real bottleneck. I'll tell you whether it's worth automating, and what it would take.

Book 30 Minutes Call