Curriculum/Large Language Models
Errors and Retries
401, 429, 5xx, and timeouts are agent outcomes. Retry only what is safe to repeat. Map errors in code, not in the prompt.
Networks fail. Vendors fail. Your key fails. The model is slow. A filter blanks the completion. An agent that treats every exception as “the model was dumb” will double-refund and then loop.
An error here is anything that is not a usable assistant turn: HTTP status codes, SDK exceptions, empty filtered content, a hang past your timeout. Map those signals to outcomes your loop already understands: retry, stop, handoff. Do not invent a fourth control plane inside the prompt (“if you see an error, try harder”).
| Signal | Typical meaning | Agent move |
|---|---|---|
| 401 / 403 | Bad, expired, or blocked key; or you may not use this model | Stop. Page a human. Do not retry. |
| 429 | Rate limit or quota | Wait (backoff), then retry read-only or idempotent calls |
| 5xx | Their server fault | Retry a few times with backoff, then hand off |
| timeout | Slow or stuck; maybe the write happened | Retry if the call was safe; else check for a duplicate |
| content filter | Finish with no useful text, or an HTTP error | Do not parse empty JSON. Handoff or refuse. |
| 400 on your body | You sent a bad schema (wrong role, huge payload) | Stop and fix the client. Retrying the same body is a loop. |
Handoff means: stop the model loop and give a human (or a ticket queue) the trace. Backoff means: wait longer each time, with a cap, so you do not hammer a 429 into a ban.
Retry only when the call is safe to repeat, or use an idempotency key. Idempotent means a second POST with the same key is a no-op or returns the same result, not a second refund. A retry that re-runs refund_customer without a key is a double refund. Time out shorter than your user will wait, then handoff. An agent that waits 120 seconds on a payment API while the user stares is already a product failure.
Safe vs unsafe
get_job is a read. Reads are usually safe to retry. refund is a write. Writes need a story: a key, a “get refund by id” check, or a human. Timeouts on writes are ambiguous: maybe the vendor ran the refund and your HTTP client gave up; maybe they never saw the request. Blind retry is how you pay twice.
The LLM call itself is usually a read in the sense that it does not charge the customer’s card. It still costs tokens if the vendor bills a partial. It still must not be retried forever. Cap attempts. Count them on the trace.
401 stops. 429 on a read can wait. A refund timeout is not “try again so the user is happy.”
Map the error before you hammer retryRun to execute this in your browser. Nothing is sent to a server.
Read the five lines. 401 always stops, even if the tool was safe — a bad key will not heal. 429 on a safe tool retries. 429 on an unsafe tool hands off (or you would need an idempotency path not shown here). Timeout on an unsafe tool hands off. Filter hands off; you do not parse empty content. That table is the policy. Put it next to the HTTP client.
Backoff: wait 1s, 2s, 4s, then cap (for example 16s). Add a small random extra so many agents do not retry in lockstep. Do not hammer 429. Do not retry 401. Do not retry 400 that is your JSON.
Filters are not `{}`
Empty filtered content is not an empty object. If you json.loads it, you will crash or invent a tool. Branch on finish reason and status first. The safety-filters lesson will go deeper. Here: treat filter as a stop, not as a parse.
A walkthrough: refund timeout
The assistant emits refund with amount 40. Your executor POSTs the payment API. The client times out at 10 seconds. You do not know if 40 left the account. decide("timeout", False) is handoff. A human (or a reconcilers job) checks the payment id. If you had an idempotency key, you could POST again with the same key and the processor would return the original refund. Without the key, code must not “just retry so the user is happy.”
The LLM retry is a different object. If the chat POST 500s before any tool runs, retrying the chat is usually safe. If the chat succeeded, you already have an assistant message, and you already ran the tool, do not replay the whole trace from step 0.
What goes wrong
- Retry everything with a generic
except Exception. You will retry auth failures and double writes. - No cap. Ten 429s with no sleep is a ban. Ten 500s is a token furnace if the vendor still bills.
- Parsing error bodies as model JSON. A vendor HTML 502 is not
{"action": "get_job"}. - Hiding the error class so the trace says “model failed.” Finance and ops need
429vstimeout. - Retrying a filter with a sneakier prompt. That is how you get banned. Change the product, not the jailbreak.
How agents use this
Put decide(error, safe) next to the HTTP client, not inside the prompt. Each tool declaration should include safe_to_retry: true/false (or “idempotent if key present”). Log the error class, attempt number, and sleep on the trace.
Circuit: if 50% of calls to a vendor 503 in five minutes, stop sending that model and fail closed or switch a pre-approved backup. That is ops, not a cleverer temperature.
Idempotency keys belong on money and deletes. Reads can retry. Writes need a story. The Tools track will go deeper on idempotency. This lesson is the LLM POST and the first hop into tools.
Tip:Time out shorter than the user’s patience. Then handoff. A hanging spinner is not “robust.”
Check your understanding