Quick Answer: There are five shapes for AI agent deployment: a long-running container on a VM, Kubernetes (Deployments for always-on agents, Jobs for single runs), serverless functions, durable workflow engines for long runs, and managed agent runtimes. Pick by run length, state and how much infrastructure you want to own. Whatever you pick, keep state outside the process, give each agent its own credentials and sandbox its tools.
AI agent deployment looks like ordinary app deployment until the first real run. A web request finishes in 200 milliseconds. An agent run can take forty minutes, call a dozen tools, open a browser, write files and wait on a model that sometimes takes a minute to answer. Then a redeploy kills it halfway through.
That's why "just put it in a container" is the start of the answer, not the end. The questions that decide your setup are how long a run lasts, where its state lives, what it's allowed to touch, and what happens when the process dies mid-task.
This guide walks through the five deployment shapes, with a minimal Dockerfile, Kubernetes manifests and a durable workflow sketch. It ends with a decision table and a pre-production checklist. We run agents in production every day at cowork.ink, where each AI coworker has its own cloud computer, so the trade-offs below are ones we deal with ourselves.
What makes AI agent deployment different?
An agent is a long-running, stateful process that acts with credentials, which a stateless web service is not. Four properties break the usual deployment assumptions:
- Run length. Runs last minutes to hours, so request timeouts, rolling restarts and scale-to-zero all become hazards.
- State. The agent carries a plan, a conversation, files and memory. Losing the process shouldn't mean losing the work.
- Side effects. It sends emails, writes to databases and calls APIs. A blind retry can send the same email twice.
- Untrusted execution. It may run code it wrote itself or browse pages that contain prompt injection, so tool execution needs a boundary.
The model call is the easy part. Hosted model APIs already handle the GPUs. What you deploy is the loop around the model: the orchestration, the tools, the state and the guardrails. If you're still deciding whether that loop should run on your hardware or someone else's, start with our local vs. cloud comparison for AI agents.
Shape 1: A long-running container on a VM
For one to a handful of agents, a container on a single VM is the simplest production setup that works. You get a fixed machine, no cold starts, no execution time limit and a filesystem that survives between runs.
Start with a small, non-root image. This is a minimal Dockerfile for a Python agent worker:
FROM python:3.12-slim
# Run as an unprivileged user, never root
RUN useradd --create-home --uid 10001 agent
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
# Writable workspace for the mounted volume
RUN mkdir -p /data && chown agent:agent /data
USER agent
ENV PYTHONUNBUFFERED=1
# A worker that pulls tasks from a queue, not a web server that holds requests open
CMD ["python", "-m", "agent.worker"]
Then run it with Docker Compose next to the services that hold its state:
services:
agent:
build: .
restart: unless-stopped
env_file: .env # model keys and tool tokens, never baked into the image
volumes:
- agent-files:/data # workspace and artifacts survive restarts
depends_on: [postgres, redis]
stop_grace_period: 10m # let an in-flight run reach a checkpoint before SIGKILL
postgres:
image: postgres:17
restart: unless-stopped
environment:
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
volumes:
- pg-data:/var/lib/postgresql/data
redis:
image: redis:7
restart: unless-stopped
volumes:
agent-files:
pg-data:
Two lines matter more than they look. restart: unless-stopped brings the agent back after a crash or reboot. stop_grace_period gives a running task time to save its progress when you deploy a new version, instead of dying mid-step.
Where it breaks: one machine is one failure domain, and scaling means a bigger VM. It's the right shape for a personal or small-team agent, which is exactly the setup our guide to self-hosting your own AI agent walks through.
Shape 2: AI agents on Kubernetes
Kubernetes is worth it when you run many agents or many runs, because it gives you scheduling, isolation, autoscaling and rollouts for free. The key is matching the Kubernetes object to the agent's lifecycle.
Deployments for always-on agents
An agent that listens for work all day (a Slack responder, a queue consumer, a scheduler) is a Deployment. Run several replicas pulling from a shared queue, and give each pod a long termination grace period so rollouts don't cut runs in half:
apiVersion: apps/v1
kind: Deployment
metadata:
name: agent-worker
spec:
replicas: 3
selector:
matchLabels: { app: agent-worker }
template:
metadata:
labels: { app: agent-worker }
spec:
serviceAccountName: agent-worker # its own identity, not the namespace default
terminationGracePeriodSeconds: 900 # finish or checkpoint the current step
containers:
- name: agent
image: registry.example.com/agent:1.4.2 # pinned tag, never :latest
envFrom:
- secretRef: { name: agent-worker-secrets }
resources:
requests: { cpu: "250m", memory: "512Mi" }
limits: { memory: "1Gi" }
Jobs for one-off agent runs
A run with a beginning and an end (a nightly research task, a migration, a batch of tickets) fits a Job. The Kubernetes Job spec gives you the three controls an agent run needs: a hard deadline, a retry budget and automatic cleanup.
apiVersion: batch/v1
kind: Job
metadata:
name: research-run-8f3a
spec:
activeDeadlineSeconds: 3600 # hard stop after 1 hour, even mid-retry
backoffLimit: 2 # at most 2 retries; agent retries are not free
ttlSecondsAfterFinished: 86400 # delete the Job and its pods a day after it ends
template:
spec:
restartPolicy: Never
containers:
- name: agent
image: registry.example.com/agent:1.4.2
args: ["run", "--task-id", "8f3a"]
activeDeadlineSeconds takes precedence over backoffLimit, so a looping agent can't burn tokens forever by failing and retrying. Keep the transcript somewhere outside the pod: once ttlSecondsAfterFinished deletes it, the pod's logs go with it.
Queues and autoscaling
Scale agent workers on queue depth, not CPU. An agent waiting on a model response uses almost no CPU, so a CPU-based autoscaler sees an idle pod while fifty tasks pile up. Event-driven autoscalers such as KEDA scale a Deployment, or spawn Jobs, from the length of a Redis, SQS or Kafka queue. Our AI agent scaling guide covers concurrency limits and rate-limit back-pressure in more depth.
Sandboxed pods for tool execution
Code the agent writes should never run in the agent's own pod. Run it in a separate, short-lived pod with no service account token, a read-only root filesystem, tight egress rules and, ideally, a sandboxed RuntimeClass such as Kata Containers. The Kubernetes SIG Apps project agent-sandbox (kubernetes-sigs/agent-sandbox on GitHub) packages this pattern as a Sandbox custom resource for isolated, stateful, single-pod workloads, with warm pools for fast allocation (its API was at v1beta1 as of November 2026).
A default rolling update sends SIGTERM and waits 30 seconds. For an agent in the middle of a 20-minute run, that is a kill. Either set a grace period longer than one step and checkpoint on SIGTERM, or drain workers (stop pulling new tasks) before the rollout starts.
Shape 3: Serverless AI agents and the timeout problem
Serverless functions are a good fit for short agent turns and a poor fit for long agent runs, because every platform caps how long one invocation can live. As of November 2026:
| Platform | Execution limit per invocation | What it means for agents |
|---|---|---|
| AWS Lambda | 15 minutes maximum | Fine for a chat turn; a long research run gets cut off |
| Google Cloud Run (services) | 60 minutes per request (5 by default) | Room for longer runs, but the client must hold the connection |
| Google Cloud Run (jobs) | Up to 168 hours (7 days) per task | Closer to a batch job than a function |
| Cloudflare Workers | No wall-clock limit while the client stays connected; up to 5 minutes of CPU on the paid plan; 15 minutes per cron run | Waiting on a model is cheap; heavy compute or long scheduled runs are not |
Sources: the AWS Lambda timeout documentation and the Google Cloud Run and Cloudflare Workers limits pages.
The upside is real: no servers, scale to zero and per-use billing. The fix for the timeout is not a bigger timeout. It's breaking the run into steps that each finish well inside the limit, with the state between them saved somewhere durable. That's the next shape.
Shape 4: Durable execution for long agent runs
A durable execution engine checkpoints every step of an agent run, so a crash, a timeout or a redeploy resumes the run instead of restarting it. For agents this matters twice over: you don't pay for completed model calls again, and a finished side effect isn't repeated on replay.
The main options, as of November 2026:
- Temporal records an ordered event history for each workflow. If a worker dies, another one replays the workflow code against that history and carries on, with retries and timeouts configured per activity.
- Inngest saves the result of each
step.run(). On replay it skips the finished steps and returns their stored results. It runs on serverless platforms without a custom runtime. - Restate keeps a journal of every step and its result, and can suspend a function on Lambda while it waits for an external event.
- AWS Lambda durable functions bring the same checkpoint-and-replay model into Lambda itself, with executions that can run for up to a year and waits that aren't billed as compute.
The shape is the same in all of them. Each model call and each side effect becomes a named step, and waiting for a human costs nothing:
// Illustrative pseudocode. Temporal, Inngest, Restate and Lambda durable
// functions each have their own SDK, but the structure is the same.
async function researchRun(task, step) {
const plan = await step.run("plan", () => llm.plan(task));
const findings = [];
for (const query of plan.queries) {
// Each search is checkpointed: a crash after query 7 resumes at query 8
findings.push(await step.run(`search-${query.id}`, () => tools.search(query)));
}
const draft = await step.run("draft", () => llm.write(task, findings));
// Pause for hours or days without holding a container
const approved = await step.waitForEvent("human-approval", { timeout: "3d" });
if (!approved) return { status: "rejected" };
// Skipped on replay once it completes. A crash mid-send retries the step,
// so pass an idempotency key the email provider can deduplicate on.
return step.run("send-report", () =>
email.send(task.owner, draft, { idempotencyKey: `report-${task.id}` }));
}
Workflow code is re-run on replay, so it must be deterministic. Model calls, tool calls, timestamps and random IDs belong inside steps, where their results are recorded. Put an LLM call outside a step and every replay will ask the model again, and get a different answer. Steps also run at least once, not exactly once: a step that crashes after its side effect but before its checkpoint is retried, so give external writes an idempotency key.
Shape 5: Managed agent runtimes
A managed agent runtime hosts your agent code, or the whole agent loop, so you skip the container platform entirely. All of the major clouds and model labs now offer one. As of November 2026:
| Runtime | What you deploy | Notable details |
|---|---|---|
| Amazon Bedrock AgentCore Runtime | Your container, any framework or model | Each session runs in its own microVM; sessions last up to 8 hours on microVMs or up to 14 days on Instances |
| Google Agent Runtime (Gemini Enterprise Agent Platform) | ADK, LangGraph, LangChain, LlamaIndex, AG2 or custom agents | Adds Sessions, Memory Bank and sandboxed Code Execution; the old Vertex AI Agent Engine docs now redirect here |
| Microsoft Foundry Agent Service | Prompt agents (configuration only) or hosted agents (your container or source zip) | Dedicated Microsoft Entra identity per hosted agent, version snapshots and rollback |
| Claude Managed Agents | Agent definition: model, prompt, tools, MCP servers, skills | Anthropic runs the harness and a cloud sandbox, or you self-host the sandbox; in beta |
| OpenAI Agents API | Agent configuration on a managed Codex harness | Saved session state; OpenAI-hosted sandbox, self-hosted sandbox or none |
The AgentCore Runtime documentation is a good illustration of the category: session isolation, long execution windows, built-in agent identity and tracing are the features you'd otherwise build yourself on Kubernetes.
The trade-off is lock-in and data location. Your agent's sessions, transcripts and memory live in the provider's cloud, on its terms. If that's a problem for your industry, read our guide to on-premise AI agents before you commit.
Which AI agent deployment shape should you choose?
Choose by run length first, then by how much operations work you're willing to own. This table summarizes the five shapes:
| Shape | Typical run length | Where state lives | Scaling | Ops burden | Best for |
|---|---|---|---|---|---|
| Container on a VM | Unlimited | Local volume plus a database | Bigger VM, by hand | Low | One to a few agents, personal or small-team use |
| Kubernetes | Unlimited (Deployments), deadline-bound (Jobs) | Persistent volumes, external database | Queue-based autoscaling | High | Many agents or runs, platform teams |
| Serverless functions | Minutes, capped per invocation | External only | Automatic, scales to zero | Low | Short, bursty agent turns |
| Durable workflow engine | Hours to days, in checkpointed steps | The engine's history or journal | Automatic per step | Medium | Long runs, human approvals, side effects that must run once |
| Managed agent runtime | Hours (provider limits apply) | Provider sessions and memory | Automatic | Lowest | Teams already on that cloud who accept its terms |
Shapes combine. A common production setup is a durable workflow engine orchestrating steps that run on Kubernetes workers, with code execution pushed to sandbox pods.
Cross-cutting concerns for every shape
The deployment shape decides where the agent runs; these six concerns decide whether it's safe to leave running. They apply to every AI agent deployment, whichever shape you pick.
State and memory persistence
Treat the container as disposable. Keep conversation and task history in a database, files on a persistent volume or object storage, and long-term memory in its own store. If an agent can't pick up after a restart, it isn't production-ready. The trade-offs between agents that remember and agents that start fresh are covered in our persistent vs. ephemeral agents guide.
Secrets and per-agent credentials
Give each agent its own identity and the narrowest scopes it needs. Don't share one admin API key across every agent. Inject secrets at runtime from a secret manager, never bake them into the image, and make sure you can revoke one agent's access without touching the others.
Sandboxing code and browser tools
Run code execution and browser sessions in a separate, short-lived environment with no access to the agent's own credentials. Restrict network egress to the domains the task needs. Treat every web page and tool output as untrusted input, because prompt injection arrives through exactly those channels.
Observability and transcripts
Log every run as a transcript: each model call, tool call, input, output, token count and cost, tied to one run ID. Standard metrics tell you the pod is alive. Only a transcript tells you why the agent emailed the wrong customer. Our AI agent observability guide covers tracing and the tools for it.
Versioning prompts and models, and rolling back
A prompt change is a code change. Version prompts, tool definitions and model IDs together with the image tag, pin model versions instead of floating aliases, and canary a new version on a slice of traffic. Let in-flight runs finish on the version they started with, then roll back by redeploying the previous tag. If you already ship through pipelines, our guide to AI agents in CI/CD shows how evaluation gates fit in.
Cost controls
Set a hard ceiling per run (steps, tokens and wall-clock time) and a daily budget per agent with an alert. A looping agent burns money silently. Kubernetes activeDeadlineSeconds, workflow timeouts and per-key spend limits at the model provider are cheap insurance.
How to deploy an AI agent to production: a checklist
Work through these steps in order before the first production run of any AI agent deployment.
Short turns go serverless; long runs need a container, a Job or a durable workflow; many agents justify Kubernetes or a managed runtime.
Database for history, volume or object storage for files, a separate store for memory. Kill the container mid-run and confirm the run resumes or fails cleanly.
Separate credentials per agent, least-privilege scopes, secrets injected at runtime, and a tested way to revoke them.
Code and browser tools run in a sandbox with restricted egress and none of the agent's credentials.
Every run logged end to end, plus alerts on failure rate, run duration and spend.
Then confirm the details:
- ✓Hard limits set: maximum steps, tokens and wall-clock time per run, and a daily budget per agent.
- ✓Side effects are idempotent: a retried step can't send the same email or create the same record twice.
- ✓Graceful shutdown tested: a rollout lets the current step finish or checkpoint instead of killing it.
- ✓Everything pinned: image tag, model version, prompt version and tool schema versioned together, with a one-command rollback.
- ✓Human approval on risky actions: payments, deletions and external messages wait for a person until the agent has earned trust.
- ✓Kill switch ready: one command or toggle stops every run of a given agent.
Self-host it, or skip deployment entirely
If you'd rather run the engine yourself, GoGogot is an open-source (MIT) AI agent written in Go that runs as a single Docker container on any Linux server, using about 10 MB of RAM at idle. It's the same long-running container shape from this guide, already built:
docker run -d --restart unless-stopped \
--name gogogot \
-e TELEGRAM_BOT_TOKEN=<token> \
-e TELEGRAM_OWNER_ID=<your-id> \
-e GOGOGOT_PROVIDER=openrouter \
-e OPENROUTER_API_KEY=<key> \
-e GOGOGOT_MODEL=deepseek/deepseek-v4-flash \
-v ./data:/data \
-v ./work:/work \
octagonlab/gogogot:latest
Memory, skills and conversations live as plain files in the mounted /data volume, so the agent survives restarts and upgrades. Full setup guide at go-go-got.com.
If you'd rather not deploy anything, cowork.ink runs the same engine as a managed service. You set up as many AI coworkers as you need, each with its own cloud computer: a real browser, a file system, an inbox and a code runtime. We handle the machines, the model bills, the updates and the uptime. You tell each coworker what it's for, connect Slack or Telegram, and give it the first task in plain language, which takes about three minutes. Finished work arrives on schedule, and every run is a transcript you can read and correct.
It's $25 a month billed yearly ($29 monthly), covering every coworker, their machines and every model. Want the same engine on your own hardware instead? See cowork.ink self-hosted.