Deploy

Deploying an agent is deploying code + policy. A prompt change can be as breaking as a schema change. Treat template edits like releases: version, review, rollback. The worker image (git SHA), the prompt bundle (p12), the tool schema bundle, and config flags (which model, whether computer-use is on) ship together. Mismatch is an incident — you already stamp versions on the job; deploy is how those versions get onto boxes.

Readiness: can the worker reach the queue, the model (or a stub), and the job store? Do not make readiness require a paid LLM call on every probe — that is a bill and a flaky restart loop. Probe a cheap health check that checks connections. Synthetic LLM checks belong on a slower cron with a stub.

This lesson is the release. Canaries and flags get their own lesson next. Kill switches after that. CI gates after that. Here we make shipping a prompt as boring as shipping a binary.

How the box actually works

Canary percent and canary SHA live in flags. pick_sha hashes the job id into a bucket. Some jobs stay on stable. A few get the new SHA. A global agents.disabled returns no SHA and admits no work. Per-tool flags (tools.refund) can disable a dangerous tool without rolling images.

Drain before a breaking worker: stop new leases on the old SHA, let in-flight slices checkpoint, then switch. Killing pods with open refunds is a retry storm you scheduled.

Owners: runtime owns images, drain, health. Prompt/ML owns bundle files in git. Ops owns flags and who can flip them. On-call owns rollback, which should be the same as “set stable_sha / prompt pointer,” not archaeology.

Prompt diffs belong in pull requests with eval results attached (the gate is two lessons away). Keep the previous template bundle on disk. Rollback is a flag flip.

Health is not intelligence. A probe that returns 200 when the queue is reachable and the job store accepts a ping is enough to restart pods. A probe that calls the frontier model will flap when the vendor is slow, spend money, and hide real process death. Put LLM synthetics on a cron with a stub and a spend cap, labeled as synthetics, not as readiness.

Drain checklist: (1) set canary/stable so new jobs skip the old SHA, (2) wait until old SHA in-flight count is 0 or only HITL-waiting jobs remain, (3) terminate old pods, (4) confirm versions on new jobs match the intended bundle.

A Friday kubectl ticket

An engineer edited the live prompt ConfigMap on Friday because “it is only text.” Refund language changed. HITL rejects climbed. There was no bundle version on jobs. Rollback was “does anyone have last week’s file?” A deploy policy later: prompts move in PRs, canary 5%, pick_sha chooses, kill refund is a flag. Friday edits in prod became a firing offense in the runbook, not a culture joke.

Health checks had been calling the frontier model every 10 seconds per pod. The bill was a sidecar. They switched to queue + disk probes. LLM synthetics moved to a 5-minute cron against a stub.

Run to execute this in your browser. Nothing is sent to a server.

job_17 stays on stable (sha-old) because its id bucket is ≥ 5. d is in the 5% canary bucket (sha-new). Refund starts allowed, then a flag flip kills refund only. Then agents.disabled makes pick_sha return None and even search is false. Killing refund, then killing all agents, is a flag flip — not a rebuild. Readiness never called a model here. Good.

What goes wrong

Prompt as a live edit. Health probes that spend. Canary percent 100 because “we are confident.” No drain. Schema and worker shipping on different days with no matrix. Flags in a dashboard nobody can diff. Readiness that requires the vendor, so a vendor blip bounce-loops your pods and makes the outage worse.

How to test it

• pick_sha fixtures: known ids → stable vs canary; disabled → None.
• can_call respects tool and global kills.
• Health endpoint does not import the LLM SDK.
• Deploy pipeline stores previous bundle; rollback test flips the pointer.
• Drain test: in-flight job checkpoints on old SHA; new jobs pick new SHA.

Attach the CI gate report to the PR that changes p12 → p13. If there is no report, it is not a deploy.

How agents use this

Treat every prompt PR like a binary PR: owners, rollback, canary. Write the rollback command in the PR template so nobody invents one during a scare. Keep images and bundles mapped in the version table you already stamp on jobs.

Readiness is connections, not intelligence. Synthetics are separate and capped.

When computer-use or refunds enter a new region, they ship off by default. Flags, not hope.

Write the rollback command in the PR template: pointer to previous bundle, previous SHA, flag names. If rollback requires a historian, you will not roll back. Keep images and bundles mapped in the version table you already stamp on jobs so the incident query and the deploy UI tell the same story.