How to Deploy AI Agents to Production

The demo works fine on your laptop. Then you ship it, and by Tuesday you're paying 400 dollars a day in token costs because a tool call started looping at 3am and nobody caught it.

Definition

Deploying an AI agent to production means moving it out of a notebook or local script and into a system that's hosted, observable, evaluated, and cost-controlled enough to serve real users with predictable reliability and bounded spend.

TL;DR

Production agents need 7 layers most prototypes skip: SLAs, infra, persistence, entry points, observability, evals, and a controlled rollout
Pick infra by workload shape. Modal or Railway for long-running agents, Vercel or Lambda for short request-response, a managed platform like LangGraph Platform if you'd rather not operate any of it
Split state into hot (Redis), durable (Postgres), and semantic (pgvector or a dedicated vector DB) instead of jamming everything into one store
Observability and evals aren't optional. Wire up Langfuse, LangSmith, or Braintrust on day one and gate every release on an eval suite
Add timeouts, retries with backoff, circuit breakers, and per-user spend caps before real traffic hits. Most production incidents are runaway loops, not bad answers

What Changes When an Agent Leaves Your Laptop

A prototype agent has one user, you. No SLA, no budget cap, no observability. Production flips every one of those. You need to know when it broke, how often, for which user, what it cost, and whether the new version actually beats the old one before you roll it out.

The other thing that changes is the failure mode. Web apps fail loudly with 500 errors. Agents fail silently with confidently wrong answers, infinite tool loops, or quietly degraded quality after a model update. Evals and traces catch those. Uptime monitoring doesn't.

The seven steps below are the order that works. Skip one and you pay for it within the first month of real traffic.

Step 1: Define Your SLAs and Failure Budget Before You Pick Tools

What to do: write down four numbers before you touch infrastructure. Target latency at p95, target success rate, max cost per request, and max cost per user per day.

Why it matters: every downstream choice falls out of these numbers. A 30-second analyst agent can run on a totally different stack than a 200ms customer-support agent. Skip the targets and you'll over-engineer the easy paths and under-engineer the dangerous ones.

A reasonable starting point for an internal tool: p95 under 8 seconds, success above 95 percent, 5 cents per request, 5 dollars per user per day. Customer-facing work changes those numbers. Whatever you pick, write it in a doc, share it with whoever owns the budget, and revisit it after a week of real traffic.

Step 2: Pick the Right Hosting Pattern for Your Workload

What to do: match your agent's runtime profile to one of three deployment patterns. Short request-response, long-running, or background queue.

Why it matters: agents aren't normal web apps. They have unpredictable latency, hold state across tool calls, and sometimes need GPU access. Put a 5-minute research agent behind Vercel's 900-second function limit and it'll work right up until it doesn't.

The three patterns:

Short request-response, under about 60 seconds total: Vercel functions, Cloudflare Workers, or AWS Lambda. Cheap, scales to zero, fine for chat agents that mostly do one or two tool calls.
Long-running or GPU-heavy, 1 to 30 minutes: Modal or Railway. Modal's Python-decorator deploy model is the closest thing to "Vercel for backends" right now, and it handles GPU and long timeouts natively.
Background or asynchronous, anything over 30 minutes or anything that needs to survive restarts: a queue plus workers (BullMQ on Redis, AWS SQS plus Lambda, or Temporal/Restate for durable execution). This is where the "agent as durable workflow" pattern lives.

Don't want to make this decision at all? A managed agent platform like LangGraph Platform, Microsoft Foundry Hosted Agents, or Vertex AI Agent Engine will pick for you, in exchange for vendor lock-in and a higher per-request cost.

Step 3: Split State Into Hot, Durable, and Semantic Layers

What to do: stop trying to make one database hold everything. Use Redis for active session state, Postgres for the system of record, and pgvector or a dedicated vector store for semantic recall.

Why it matters: a single agent run touches three very different storage workloads. Tool call scratch space and partial results need millisecond reads. Conversation history, user prefs, and audit logs need ACID durability. Past experiences and document chunks need similarity search. One database does at most two of these well.

The pattern that's become standard in 2026 is the hybrid write-through model. Active turn state lives in Redis with a TTL. A background worker flushes completed turns to Postgres and computes embeddings for the chunks worth recalling, storing them in pgvector. When the agent wakes up for a new turn, it hydrates working memory from Redis if the session is hot, falls back to Postgres if it's cold, and pulls episodic context from pgvector by similarity.

Two gotchas worth knowing. Redis with default RDB persistence can lose data up to the last snapshot interval, so never treat it as the system of record. And don't store secrets or PII in vector indexes. Embed a reference ID instead, and look up the actual content from Postgres at retrieval time.

Step 4: Deploy the Entry Points (HTTP, Webhook, Queue)

What to do: expose your agent through whichever entry points your product needs, and put authentication, rate limits, and request validation in front of every one of them.

Why it matters: agents are expensive to run. An unauthenticated webhook that spawns an agent is a denial-of-wallet attack waiting to happen. I have personally seen a $12,000 AWS bill from one exposed endpoint over a long weekend.

A solid baseline:

Authentication on every entry point. API key for server-to-server, signed JWT for user sessions, HMAC verification for webhooks
Per-user rate limit at the edge (Upstash Redis, Cloudflare, or your gateway)
Request schema validation with Zod or Pydantic. Reject malformed payloads before they ever reach an LLM call
Idempotency keys on any endpoint that can mutate state. Agents retry, and without idempotency, retries cause duplicate writes

Wrap the agent itself in a thin handler that does auth, validates input, generates a trace ID, kicks off the run, and returns. The handler should never contain agent logic.

Step 5: Wire Observability Before You Take Real Traffic

What to do: instrument every LLM call and tool call with traces, costs, latency, and inputs and outputs. Pick one observability platform and standardize on it from day one.

Why it matters: when your agent misbehaves in production, you need to replay the exact run that caused it. Without traces, you're reading customer screenshots and guessing. With them, you click into the trace and see the bad tool argument that started the loop.

The three platforms worth looking at in 2026:

Langfuse: open-source, MIT-licensed, self-hostable. Best choice if you care about data residency or want to avoid another SaaS bill
LangSmith: tightest integration with LangChain and LangGraph, including state-diff tracing. Default if you're already on that stack
Braintrust: eval-first platform with CI/CD deployment blocking. Best if your team treats evals as a primary engineering workflow

Whichever you pick, capture five things on every run: the trace ID, full input, full output, token counts and dollar cost per call, and total wall-clock latency. That's the minimum to debug anything.

Step 6: Build an Eval Suite and Gate Releases on It

What to do: build a dataset of 30 to 100 real user inputs with expected outcomes, run your agent against it on every change, and block deploys that regress on key metrics.

Why it matters: the failure mode of agents isn't a 500 error, it's a quality regression. You change a prompt, fix one bug, and silently break three other behaviors. Skip evals and you find out from a customer complaint two weeks later. With evals, the deploy doesn't ship.

A starter eval suite has three tiers. Unit-level checks on tool selection: given input X, does the agent call tool Y with arguments Z? Outcome-level checks on task success: did the final answer match the expected outcome on a graded rubric? End-to-end checks on regressions: rerun the last 30 days of real production traces against the new version and flag anything that scores lower than before.

Use LLM-as-judge for the graded rubric, but anchor it. Give it the rubric, give it 3 to 5 hand-graded examples in the prompt, and require a score plus a reason. Spot-check 10 percent of judge scores manually each week or the rubric drifts.

Run the suite in CI. Braintrust, LangSmith, and Langfuse all support CI integration. If you self-host, a GitHub Action that runs the suite and posts results to the PR is enough to start.

Step 7: Roll Out With Retries, Circuit Breakers, and Cost Caps

What to do: deploy the new version alongside the old one, route 5 to 10 percent of traffic to it, monitor for an hour minimum, then expand. Wrap every external call in retries with exponential backoff, circuit breakers, and a hard spend cap per user per day.

Why it matters: most production agent incidents aren't bad answers. They're runaway loops, third-party API outages, or one user accidentally running up 8,000 dollars of inference because a prompt injection convinced the agent to call a tool 400 times.

The minimum protection layer:

Retries with exponential backoff and jitter on every external API. Cap retries at 3. Don't retry on 4xx errors except 408 and 429
Circuit breakers on every tool. After N consecutive failures, open the circuit and short-circuit subsequent calls for a cool-off window. The agent gets an error back and can either work around it or fail gracefully
Per-run iteration cap. Most agents should complete in under 20 reasoning steps. Cap at 30 and kill the run if it exceeds. Log it and alert on it
Per-user daily spend cap, enforced before each LLM call by checking a Redis counter. When the cap is hit, return a friendly error and notify the user
Canary deployment. Route a small percentage of traffic to the new version. Compare success rate, latency, and cost-per-request to the baseline, and roll forward only if all three hold or improve

Get those in place and the worst-case incident is a contained one. Skip them and the worst case is a five-figure bill.

Warning

Never wire OpenAI or Anthropic keys directly into client code, browser apps, or Slack bots. Always proxy through a server you control where you can enforce auth, rate limits, and spend caps.

Hosting Comparison: Where to Actually Deploy

Use this as a starting point, then validate it against your own latency and cost targets from step 1.

Platform	Best For	Max Runtime	Starting Price
Modal	Long-running, GPU, Python-first agents	24 hours	Pay per second of compute
Vercel Functions	Short chat agents next to a Next.js app	900 seconds (Fluid)	Free tier, then usage-based
AWS Lambda	Event-driven agents in AWS-native stacks	900 seconds	Pay per request and GB-second
Railway	Long-running Node/Python agents with WebSockets	No hard limit	5 dollars per month plus usage
LangGraph Platform	Managed deploys for LangGraph agents	Long-running supported	Usage-based, free dev tier
Cloudflare Workers	Edge-deployed, low-latency agents	30 minutes (paid plan)	Free tier, then 5 dollars per month

Starting fresh in 2026 with no strong platform preference already? Modal for long-running work, Vercel for the user-facing chat surface, and Langfuse for traces is a stack you can be productive on by the end of day one.

Putting the Stack Together

A real production deployment looks like this. Vercel or Modal hosts the agent runtime. Postgres on Neon or Supabase holds durable state. Upstash Redis holds session state and rate-limit counters. Langfuse or LangSmith captures every trace. Braintrust runs the eval suite in CI. A canary on 10 percent of traffic guards every deploy. Per-user spend caps and circuit breakers run on every request.

That stack costs roughly 50 to 200 dollars a month at low volume, before LLM tokens. It's cheap insurance against the mistake that costs four figures overnight.

Before you go live, make sure the design side is solid, you've picked a framework you're comfortable maintaining, and you've built the guardrails this stack assumes: iteration caps, spend limits, and circuit breakers on every tool call.

What is the difference between deploying an AI agent and deploying a normal web app?

A web app handles short, stateless requests with predictable latency and fails loudly when something breaks. An AI agent runs longer, holds state across tool calls, has unpredictable latency and cost per request, and tends to fail silently with wrong answers or runaway loops. That means production agents need traces, evals, per-user spend caps, and circuit breakers that normal web apps usually skip.

Where should I host an AI agent in production?

Pick by workload shape. Short chat agents under 60 seconds run well on Vercel, Cloudflare Workers, or AWS Lambda. Longer-running agents up to 30 minutes are best on Modal or Railway, where Modal handles GPU and long timeouts natively. Anything truly long-running or that needs to survive restarts belongs in a queue plus worker pattern using Temporal, Restate, or BullMQ.

Do I need a vector database to run an agent in production?

Only if your agent uses semantic recall over past conversations or documents. Many production agents work fine with Postgres for durable state and Redis for session state, and never need a vector store. If you do need semantic search, pgvector inside the same Postgres you already run is the simplest starting point. Reach for a dedicated vector DB like Pinecone or Weaviate only when query latency or scale requires it.

How do I monitor an AI agent in production?

Use a dedicated LLM observability platform. Langfuse if you want open-source and self-hosted, LangSmith if you're on the LangChain or LangGraph stack, Braintrust if you want eval-driven CI/CD gating. Capture the full input, full output, token cost, latency, and a unique trace ID on every run. Standard application monitoring tools like Datadog or New Relic miss the LLM-specific signals you actually need to debug agents.

How do I stop an AI agent from running up a huge API bill?

Layer four controls. Per-run iteration caps that kill the agent after 20 to 30 reasoning steps. Per-user daily spend caps enforced in Redis before every LLM call. Circuit breakers on tools so a failing API stops getting called after a few consecutive errors. Authentication and rate limits on every entry point so unauthenticated requests can't trigger LLM calls. Get those four in place and the worst-case incident is contained instead of catastrophic.

When should I use a managed agent platform versus building my own deployment?

Use a managed platform like LangGraph Platform, Microsoft Foundry Hosted Agents, or Vertex AI Agent Engine when you want to ship fast, don't have a platform team, and are fine with vendor lock-in plus a higher per-request cost. Build your own on Modal or Railway plus Postgres plus Langfuse when you have specific requirements around latency, cost, data residency, or custom infrastructure the managed platforms don't support. Most teams should start managed and migrate later, only if the per-request economics force it.