Contain, Then See
After the flag flips: traces for the blast radius, classify policy vs tool vs vendor vs injection vs HITL bypass, then rollback and rotate if keys leaked. Do not delete traces.
Minute 5–20 is see. Pull traces for the blast radius (tenant, tool, time window). Distinguish policy bug (wrong prompt / state) vs tool bug (authz miss) vs vendor outage vs injection vs HITL bypass. Check whether the human gate was skipped.
Minute 20+ is mitigate: rollback worker SHA / prompt bundle; rotate keys if traces or prompts may have leaked secrets; replay read-only to confirm the fix on recorded observations; customer comms with facts, not vibes. Do not delete traces to “save face.” You will need them for customers, auditors, and the golden you are about to write.
Containment without classification is how you leave the wrong flag off for a week. Classification without containment is how you write a beautiful RCA while refunds continue.
How the box actually works
A small classifier over the trace is enough to pick the first lever. Humans still confirm.
| Trace picture | Class | First lever (after contain) |
|---|---|---|
wire tool | policy or injection | Keep dangerous tools off; inspect observations for injected text |
get_invoice two tenants on one job | tool authz | Disable retrieval if needed; patch filter |
VENDOR_503 | vendor | Breaker already; slow enqueue; do not mass-retry huge contexts |
refund without hitl_approved | HITL bypass | Refund stays off until gate is real |
| Cost explosion, tools look normal | cap / loop | Job cap, kill new jobs, trim |
| Multi-agent ping-pong | hop cap | Disable handoff |
| Injection via PDF | tool family | Disable that family; strip observations |
Do not delete traces. Replay write tools as stubs.
Classify after the flagBlast radius query: tenant, tool name, time window, versions stamp. Export redacted traces for the incident doc.
Owners: on-call classifies with a buddy. Security joins on leak/injection. Vendor comms if 503 is them. Do not let five people each “just tweak” a different flag.
Replay read-only: recorded observations, fake or frozen model, write tools stubbed. If you replay live refunds you will make a second incident.
Classification is a sorting hat for levers, not a court verdict. When two classes could fit, pick the stricter contain (more flags off) until you know. A vendor 503 plus a HITL bypass in the same hour is two incidents; do not average them.
Keep traces. Restrict access. Redact for the incident doc. Deleting traces to save face is how you fail the audit and the golden. The blast-radius query is tenant + tool + window + versions; export that set once into the shared doc so five people are not running five slightly different greps.
A two-tenant-read ticket
Traces showed get_invoice for Acme and Beta on one job. Someone argued it was “a creative model.” Classification was tool_authz. Containment: disable retrieval (and invoice) until the filter shipped. They did not delete traces. Legal needed the ids. The golden was the tempting query from the tenant lesson. Rollback was not required; the worker SHA was fine. The tool was not.
A parallel false path: deleting traces to save face. That turns a containable leak into an unprovable one. Keep traces. Redact. Restrict access. Do not incinerate evidence.
Run to execute this in your browser. Nothing is sent to a server.
storm is hitl_bypass (refund, no human flag). vendor is vendor because of VENDOR_503. inject is policy_or_injection because of wire. Refund without HITL is a bypass. Wire is policy or injection. 503 is the vendor. Different first levers. needs_human is the honest default when the picture does not match.
What goes wrong
Tweaking prompts while classifying. Deleting traces. Replaying write tools live. One channel with twenty theories and no blast-radius query. Treating injection as only a wording problem while the PDF tool still runs. Rolling back the worker when the tool authz is the bug (you will “fix” nothing).
A classification of needs_human that sits for an hour with no human is contain-and-freeze, not see. Page the owner named on the box. If you cannot name the owner, that gap is the incident too.
How to test it
- Classifier fixtures: storm, vendor, wire, two-tenant invoice, unknown.
- Blast-radius query in staging returns the seeded jobs and only those.
- Replay harness refuses live write tools.
- Access log: incident traces still readable after flags flip.
Seed a two-tenant invoice trace and assert classify returns tool_authz, not needs_human. Seed a 503 and assert you do not roll back the worker in the runbook’s next step. The classifier is a small function so you can test it without a cluster.
How agents use this
Common first levers: cost explosion → job cap + kill new jobs. Refund storm → refund flag + ledger audit. Cross-tenant read → disable retrieval, then patch authz. Multi-agent ping-pong → hop cap, disable handoff. Injection via PDF → disable that tool family, strip observations.
Write those levers next to classify in the runbook. After see, mitigate with rollback and rotation if the class needs it. Then the golden.
Rollback the worker when the class is policy/code on the worker. Patch the tool when the class is authz. Rotate keys when traces or prompts may have contained secrets. Do not roll back a healthy SHA because the model “felt off.” The versions stamp tells you what to roll.
If classification is policy_or_injection, keep the dangerous tool off while you read observations for injected instructions. If it is vendor, do not also rewrite the prompt. Wrong mitigations stack into a second outage.
Write the blast-radius query as a saved search with placeholders for tenant, tool, and window. On-call should paste three values, not invent SQL. If the query takes more than a minute to run, you will skip see.
Check your understanding