AI Agent Deployment: Docker, Kubernetes, Serverless & More

AI agent deployment compared: Docker, Kubernetes, serverless, durable workflows and managed runtimes, with code and a CHECKLIST. Pick yours now.

Quick Answer: There are five shapes for AI agent deployment: a long-running container on a VM, Kubernetes (Deployments for always-on agents, Jobs for single runs), serverless functions, durable workflow engines for long runs, and managed agent runtimes. Pick by run length, state and how much infrastructure you want to own. Whatever you pick, keep state outside the process, give each agent its own credentials and sandbox its tools.


AI agent deployment looks like ordinary app deployment until the first real run. A web request finishes in 200 milliseconds. An agent run can take forty minutes, call a dozen tools, open a browser, write files and wait on a model that sometimes takes a minute to answer. Then a redeploy kills it halfway through.

That's why "just put it in a container" is the start of the answer, not the end. The questions that decide your setup are how long a run lasts, where its state lives, what it's allowed to touch, and what happens when the process dies mid-task.

This guide walks through the five deployment shapes, with a minimal Dockerfile, Kubernetes manifests and a durable workflow sketch. It ends with a decision table and a pre-production checklist. We run agents in production every day at cowork.ink, where each AI coworker has its own cloud computer, so the trade-offs below are ones we deal with ourselves.


What makes AI agent deployment different?

An agent is a long-running, stateful process that acts with credentials, which a stateless web service is not. Four properties break the usual deployment assumptions:

  • Run length. Runs last minutes to hours, so request timeouts, rolling restarts and scale-to-zero all become hazards.
  • State. The agent carries a plan, a conversation, files and memory. Losing the process shouldn't mean losing the work.
  • Side effects. It sends emails, writes to databases and calls APIs. A blind retry can send the same email twice.
  • Untrusted execution. It may run code it wrote itself or browse pages that contain prompt injection, so tool execution needs a boundary.

The model call is the easy part. Hosted model APIs already handle the GPUs. What you deploy is the loop around the model: the orchestration, the tools, the state and the guardrails. If you're still deciding whether that loop should run on your hardware or someone else's, start with our local vs. cloud comparison for AI agents.


Shape 1: A long-running container on a VM

For one to a handful of agents, a container on a single VM is the simplest production setup that works. You get a fixed machine, no cold starts, no execution time limit and a filesystem that survives between runs.

Start with a small, non-root image. This is a minimal Dockerfile for a Python agent worker:

FROM python:3.12-slim

# Run as an unprivileged user, never root
RUN useradd --create-home --uid 10001 agent
WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .

# Writable workspace for the mounted volume
RUN mkdir -p /data && chown agent:agent /data

USER agent
ENV PYTHONUNBUFFERED=1

# A worker that pulls tasks from a queue, not a web server that holds requests open
CMD ["python", "-m", "agent.worker"]

Then run it with Docker Compose next to the services that hold its state:

services:
  agent:
    build: .
    restart: unless-stopped
    env_file: .env            # model keys and tool tokens, never baked into the image
    volumes:
      - agent-files:/data      # workspace and artifacts survive restarts
    depends_on: [postgres, redis]
    stop_grace_period: 10m     # let an in-flight run reach a checkpoint before SIGKILL

  postgres:
    image: postgres:17
    restart: unless-stopped
    environment:
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    volumes:
      - pg-data:/var/lib/postgresql/data

  redis:
    image: redis:7
    restart: unless-stopped

volumes:
  agent-files:
  pg-data:

Two lines matter more than they look. restart: unless-stopped brings the agent back after a crash or reboot. stop_grace_period gives a running task time to save its progress when you deploy a new version, instead of dying mid-step.

Where it breaks: one machine is one failure domain, and scaling means a bigger VM. It's the right shape for a personal or small-team agent, which is exactly the setup our guide to self-hosting your own AI agent walks through.


Shape 2: AI agents on Kubernetes

Kubernetes is worth it when you run many agents or many runs, because it gives you scheduling, isolation, autoscaling and rollouts for free. The key is matching the Kubernetes object to the agent's lifecycle.

Deployments for always-on agents

An agent that listens for work all day (a Slack responder, a queue consumer, a scheduler) is a Deployment. Run several replicas pulling from a shared queue, and give each pod a long termination grace period so rollouts don't cut runs in half:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: agent-worker
spec:
  replicas: 3
  selector:
    matchLabels: { app: agent-worker }
  template:
    metadata:
      labels: { app: agent-worker }
    spec:
      serviceAccountName: agent-worker      # its own identity, not the namespace default
      terminationGracePeriodSeconds: 900    # finish or checkpoint the current step
      containers:
        - name: agent
          image: registry.example.com/agent:1.4.2   # pinned tag, never :latest
          envFrom:
            - secretRef: { name: agent-worker-secrets }
          resources:
            requests: { cpu: "250m", memory: "512Mi" }
            limits:   { memory: "1Gi" }

Jobs for one-off agent runs

A run with a beginning and an end (a nightly research task, a migration, a batch of tickets) fits a Job. The Kubernetes Job spec gives you the three controls an agent run needs: a hard deadline, a retry budget and automatic cleanup.

apiVersion: batch/v1
kind: Job
metadata:
  name: research-run-8f3a
spec:
  activeDeadlineSeconds: 3600     # hard stop after 1 hour, even mid-retry
  backoffLimit: 2                 # at most 2 retries; agent retries are not free
  ttlSecondsAfterFinished: 86400  # delete the Job and its pods a day after it ends
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: agent
          image: registry.example.com/agent:1.4.2
          args: ["run", "--task-id", "8f3a"]

activeDeadlineSeconds takes precedence over backoffLimit, so a looping agent can't burn tokens forever by failing and retrying. Keep the transcript somewhere outside the pod: once ttlSecondsAfterFinished deletes it, the pod's logs go with it.

Queues and autoscaling

Scale agent workers on queue depth, not CPU. An agent waiting on a model response uses almost no CPU, so a CPU-based autoscaler sees an idle pod while fifty tasks pile up. Event-driven autoscalers such as KEDA scale a Deployment, or spawn Jobs, from the length of a Redis, SQS or Kafka queue. Our AI agent scaling guide covers concurrency limits and rate-limit back-pressure in more depth.

Sandboxed pods for tool execution

Code the agent writes should never run in the agent's own pod. Run it in a separate, short-lived pod with no service account token, a read-only root filesystem, tight egress rules and, ideally, a sandboxed RuntimeClass such as Kata Containers. The Kubernetes SIG Apps project agent-sandbox (kubernetes-sigs/agent-sandbox on GitHub) packages this pattern as a Sandbox custom resource for isolated, stateful, single-pod workloads, with warm pools for fast allocation (its API was at v1beta1 as of November 2026).

The rollout trap

A default rolling update sends SIGTERM and waits 30 seconds. For an agent in the middle of a 20-minute run, that is a kill. Either set a grace period longer than one step and checkpoint on SIGTERM, or drain workers (stop pulling new tasks) before the rollout starts.


Shape 3: Serverless AI agents and the timeout problem

Serverless functions are a good fit for short agent turns and a poor fit for long agent runs, because every platform caps how long one invocation can live. As of November 2026:

PlatformExecution limit per invocationWhat it means for agents
AWS Lambda15 minutes maximumFine for a chat turn; a long research run gets cut off
Google Cloud Run (services)60 minutes per request (5 by default)Room for longer runs, but the client must hold the connection
Google Cloud Run (jobs)Up to 168 hours (7 days) per taskCloser to a batch job than a function
Cloudflare WorkersNo wall-clock limit while the client stays connected; up to 5 minutes of CPU on the paid plan; 15 minutes per cron runWaiting on a model is cheap; heavy compute or long scheduled runs are not

Sources: the AWS Lambda timeout documentation and the Google Cloud Run and Cloudflare Workers limits pages.

The upside is real: no servers, scale to zero and per-use billing. The fix for the timeout is not a bigger timeout. It's breaking the run into steps that each finish well inside the limit, with the state between them saved somewhere durable. That's the next shape.


Shape 4: Durable execution for long agent runs

A durable execution engine checkpoints every step of an agent run, so a crash, a timeout or a redeploy resumes the run instead of restarting it. For agents this matters twice over: you don't pay for completed model calls again, and a finished side effect isn't repeated on replay.

The main options, as of November 2026:

  • Temporal records an ordered event history for each workflow. If a worker dies, another one replays the workflow code against that history and carries on, with retries and timeouts configured per activity.
  • Inngest saves the result of each step.run(). On replay it skips the finished steps and returns their stored results. It runs on serverless platforms without a custom runtime.
  • Restate keeps a journal of every step and its result, and can suspend a function on Lambda while it waits for an external event.
  • AWS Lambda durable functions bring the same checkpoint-and-replay model into Lambda itself, with executions that can run for up to a year and waits that aren't billed as compute.

The shape is the same in all of them. Each model call and each side effect becomes a named step, and waiting for a human costs nothing:

// Illustrative pseudocode. Temporal, Inngest, Restate and Lambda durable
// functions each have their own SDK, but the structure is the same.
async function researchRun(task, step) {
  const plan = await step.run("plan", () => llm.plan(task));

  const findings = [];
  for (const query of plan.queries) {
    // Each search is checkpointed: a crash after query 7 resumes at query 8
    findings.push(await step.run(`search-${query.id}`, () => tools.search(query)));
  }

  const draft = await step.run("draft", () => llm.write(task, findings));

  // Pause for hours or days without holding a container
  const approved = await step.waitForEvent("human-approval", { timeout: "3d" });
  if (!approved) return { status: "rejected" };

  // Skipped on replay once it completes. A crash mid-send retries the step,
  // so pass an idempotency key the email provider can deduplicate on.
  return step.run("send-report", () =>
    email.send(task.owner, draft, { idempotencyKey: `report-${task.id}` }));
}
Replay has a rule

Workflow code is re-run on replay, so it must be deterministic. Model calls, tool calls, timestamps and random IDs belong inside steps, where their results are recorded. Put an LLM call outside a step and every replay will ask the model again, and get a different answer. Steps also run at least once, not exactly once: a step that crashes after its side effect but before its checkpoint is retried, so give external writes an idempotency key.


Shape 5: Managed agent runtimes

A managed agent runtime hosts your agent code, or the whole agent loop, so you skip the container platform entirely. All of the major clouds and model labs now offer one. As of November 2026:

RuntimeWhat you deployNotable details
Amazon Bedrock AgentCore RuntimeYour container, any framework or modelEach session runs in its own microVM; sessions last up to 8 hours on microVMs or up to 14 days on Instances
Google Agent Runtime (Gemini Enterprise Agent Platform)ADK, LangGraph, LangChain, LlamaIndex, AG2 or custom agentsAdds Sessions, Memory Bank and sandboxed Code Execution; the old Vertex AI Agent Engine docs now redirect here
Microsoft Foundry Agent ServicePrompt agents (configuration only) or hosted agents (your container or source zip)Dedicated Microsoft Entra identity per hosted agent, version snapshots and rollback
Claude Managed AgentsAgent definition: model, prompt, tools, MCP servers, skillsAnthropic runs the harness and a cloud sandbox, or you self-host the sandbox; in beta
OpenAI Agents APIAgent configuration on a managed Codex harnessSaved session state; OpenAI-hosted sandbox, self-hosted sandbox or none

The AgentCore Runtime documentation is a good illustration of the category: session isolation, long execution windows, built-in agent identity and tracing are the features you'd otherwise build yourself on Kubernetes.

The trade-off is lock-in and data location. Your agent's sessions, transcripts and memory live in the provider's cloud, on its terms. If that's a problem for your industry, read our guide to on-premise AI agents before you commit.


Which AI agent deployment shape should you choose?

Choose by run length first, then by how much operations work you're willing to own. This table summarizes the five shapes:

ShapeTypical run lengthWhere state livesScalingOps burdenBest for
Container on a VMUnlimitedLocal volume plus a databaseBigger VM, by handLowOne to a few agents, personal or small-team use
KubernetesUnlimited (Deployments), deadline-bound (Jobs)Persistent volumes, external databaseQueue-based autoscalingHighMany agents or runs, platform teams
Serverless functionsMinutes, capped per invocationExternal onlyAutomatic, scales to zeroLowShort, bursty agent turns
Durable workflow engineHours to days, in checkpointed stepsThe engine's history or journalAutomatic per stepMediumLong runs, human approvals, side effects that must run once
Managed agent runtimeHours (provider limits apply)Provider sessions and memoryAutomaticLowestTeams already on that cloud who accept its terms

Shapes combine. A common production setup is a durable workflow engine orchestrating steps that run on Kubernetes workers, with code execution pushed to sandbox pods.


Cross-cutting concerns for every shape

The deployment shape decides where the agent runs; these six concerns decide whether it's safe to leave running. They apply to every AI agent deployment, whichever shape you pick.

State and memory persistence

Treat the container as disposable. Keep conversation and task history in a database, files on a persistent volume or object storage, and long-term memory in its own store. If an agent can't pick up after a restart, it isn't production-ready. The trade-offs between agents that remember and agents that start fresh are covered in our persistent vs. ephemeral agents guide.

Secrets and per-agent credentials

Give each agent its own identity and the narrowest scopes it needs. Don't share one admin API key across every agent. Inject secrets at runtime from a secret manager, never bake them into the image, and make sure you can revoke one agent's access without touching the others.

Sandboxing code and browser tools

Run code execution and browser sessions in a separate, short-lived environment with no access to the agent's own credentials. Restrict network egress to the domains the task needs. Treat every web page and tool output as untrusted input, because prompt injection arrives through exactly those channels.

Observability and transcripts

Log every run as a transcript: each model call, tool call, input, output, token count and cost, tied to one run ID. Standard metrics tell you the pod is alive. Only a transcript tells you why the agent emailed the wrong customer. Our AI agent observability guide covers tracing and the tools for it.

Versioning prompts and models, and rolling back

A prompt change is a code change. Version prompts, tool definitions and model IDs together with the image tag, pin model versions instead of floating aliases, and canary a new version on a slice of traffic. Let in-flight runs finish on the version they started with, then roll back by redeploying the previous tag. If you already ship through pipelines, our guide to AI agents in CI/CD shows how evaluation gates fit in.

Cost controls

Set a hard ceiling per run (steps, tokens and wall-clock time) and a daily budget per agent with an alert. A looping agent burns money silently. Kubernetes activeDeadlineSeconds, workflow timeouts and per-key spend limits at the model provider are cheap insurance.


How to deploy an AI agent to production: a checklist

Work through these steps in order before the first production run of any AI agent deployment.

1
Pick the shape by run length

Short turns go serverless; long runs need a container, a Job or a durable workflow; many agents justify Kubernetes or a managed runtime.

2
Move state out of the process

Database for history, volume or object storage for files, a separate store for memory. Kill the container mid-run and confirm the run resumes or fails cleanly.

3
Give the agent its own identity

Separate credentials per agent, least-privilege scopes, secrets injected at runtime, and a tested way to revoke them.

4
Isolate tool execution

Code and browser tools run in a sandbox with restricted egress and none of the agent's credentials.

5
Turn on transcripts and alerts

Every run logged end to end, plus alerts on failure rate, run duration and spend.

Then confirm the details:

  • ✓Hard limits set: maximum steps, tokens and wall-clock time per run, and a daily budget per agent.
  • ✓Side effects are idempotent: a retried step can't send the same email or create the same record twice.
  • ✓Graceful shutdown tested: a rollout lets the current step finish or checkpoint instead of killing it.
  • ✓Everything pinned: image tag, model version, prompt version and tool schema versioned together, with a one-command rollback.
  • ✓Human approval on risky actions: payments, deletions and external messages wait for a person until the agent has earned trust.
  • ✓Kill switch ready: one command or toggle stops every run of a given agent.

Self-host it, or skip deployment entirely

If you'd rather run the engine yourself, GoGogot is an open-source (MIT) AI agent written in Go that runs as a single Docker container on any Linux server, using about 10 MB of RAM at idle. It's the same long-running container shape from this guide, already built:

docker run -d --restart unless-stopped \
  --name gogogot \
  -e TELEGRAM_BOT_TOKEN=<token> \
  -e TELEGRAM_OWNER_ID=<your-id> \
  -e GOGOGOT_PROVIDER=openrouter \
  -e OPENROUTER_API_KEY=<key> \
  -e GOGOGOT_MODEL=deepseek/deepseek-v4-flash \
  -v ./data:/data \
  -v ./work:/work \
  octagonlab/gogogot:latest

Memory, skills and conversations live as plain files in the mounted /data volume, so the agent survives restarts and upgrades. Full setup guide at go-go-got.com.

If you'd rather not deploy anything, cowork.ink runs the same engine as a managed service. You set up as many AI coworkers as you need, each with its own cloud computer: a real browser, a file system, an inbox and a code runtime. We handle the machines, the model bills, the updates and the uptime. You tell each coworker what it's for, connect Slack or Telegram, and give it the first task in plain language, which takes about three minutes. Finished work arrives on schedule, and every run is a transcript you can read and correct.

It's $25 a month billed yearly ($29 monthly), covering every coworker, their machines and every model. Want the same engine on your own hardware instead? See cowork.ink self-hosted.

Frequently Asked Questions

How do you deploy an AI agent to production?
Package the agent as a container, keep its state outside the process, give it its own scoped credentials, run tool execution in a sandbox and log every run as a transcript. Then pick a shape that fits the run length: a container on a VM, Kubernetes, serverless with a durable workflow engine, or a managed agent runtime.
Can AI agents run on serverless functions?
Yes, for short runs. As of November 2026 a standard AWS Lambda function stops at 15 minutes and a Cloud Run request at 60 minutes, so longer agent runs need a durable workflow layer (Lambda durable functions, Temporal, Inngest or Restate) that checkpoints each step and resumes after the timeout.
Should you deploy AI agents on Kubernetes?
Kubernetes pays off when you run many agents or many runs: Deployments for always-on agent workers, Jobs for one-off runs, queue-based autoscaling and separate sandbox pods for code execution. For one or two agents, a container on a single VM is simpler. See our guide to [AI agent scaling](/blog/ai-agent-scaling/) for when the extra machinery starts to matter.
Where can I host an AI agent without managing servers?
Managed agent runtimes host the container or the whole agent loop for you: Amazon Bedrock AgentCore Runtime, Google's Agent Runtime in Gemini Enterprise Agent Platform, Microsoft Foundry Agent Service, Claude Managed Agents (beta) and the OpenAI Agents API. If you want finished work rather than an endpoint, cowork.ink runs the machines, models and uptime for you.
How do AI agents keep state between runs?
Production agents store state outside the container: conversation and task history in a database, files on a persistent volume or object storage, and long-term memory in its own store. That way a crash, a redeploy or a scale-down doesn't wipe what the agent knows. Our [persistent vs. ephemeral agents](/blog/persistent-vs-ephemeral-ai-agents/) guide covers the trade-off.
Home Blog Company