Retries and Timeouts

A retry means try the same call again. Retries save you from flaky networks. They also multiply cost if you retry the wrong thing. A 400 with a missing field will 400 forever. A 429 might succeed if you wait.

Retry a timeout (no answer in time). Retry 429 (too many requests). Do not retry 400 (your request is wrong). The same body will fail again. Do not retry 401 either — fix the key.

This is still Python: sets of statuses, a for-loop with a cap, a list of wait numbers you print instead of sleeping. The policy is the point. time.sleep would freeze this page.

Retry only some failures

Max tries is a hard cap. Three is a common default. After that, fail. An agent that retries forever is a bill. Max tries applies per call, not per agent lifetime. The agent loop has its own max_steps. Nested unbounded retries inside each step are how 8 steps become 800 HTTP calls.

Idempotency matters. Retrying GET search is usually safe. Retrying “charge the card” may double-charge. This lesson’s fake API is search-like. Do not blindly retry every tool.

Backoff: a list of waits

Backoff means wait longer each time. You do not need a fancy formula. Keep a list of wait numbers. Print the wait. Do not sleep for a long time in this editor — it would freeze the page.

waits = [0.5, 1.0, 2.0]
class="tok-c"># try 1 fails -> would wait 0.5
class="tok-c"># try 2 fails -> would wait 1.0
class="tok-c"># try 3 fails -> stop
print(waits)

On a laptop you might time.sleep(wait). Here we only print would wait. Logging the wait is still useful in production. Then a trace explains the gap between two timestamps.

If you have more tries than waits, stop at max tries anyway. Do not invent waits with a while True. Index waits[attempt - 1] only when you will retry.

A circuit: stop after N fails

A circuit counts fails in a row. After N fails, you open the circuit: stop calling that API for a while. That protects you from a dead server. A success closes it and resets the count.

Think of a fuse. Too many sparks, the fuse pops. You do not keep sending sparks. In code, open_ is a bool (the name open would shadow the builtin). When open, skip the call. You can later add a cooldown timestamp. A skip loop is enough to see the idea.

A circuit is per dependency: the search API, not the whole agent. One broken tool should not freeze finish if finish is local.

Jitter means adding a little randomness to the wait so many agents do not retry on the same second. You do not need it in this box. On a laptop, a tiny random add is enough. Still cap tries. Jitter without a cap is still a bill.

Count attempts from 1 in logs. Humans say “third try.” range(1, max_tries + 1) matches that speech. If the script of fake statuses is shorter than max tries, stop when the script ends. Tests pass a list of statuses. Production passes a function that hits the network. Same loop.

Do not retry finish. Finish is local. Do not retry a parser error. The body is wrong. Retry is for “the other side blinked.” If you cannot tell those apart, default to no retry and log why.

500 is a maybe. One retry is reasonable. Three retries on a persistently 500 API is paying for their outage. After max tries, return an error dict and let the agent loop decide to finish or try a different tool. The circuit is for “this API is dead for this run.” It is not a substitute for max_steps on the agent.

Common mistakes

• Retrying 400/401.
• No max tries.
• Sleeping 30s in a demo box.
• Retrying non-idempotent writes.
• A circuit that never resets on success (here, success resets fails; once open, this demo stays open — production would cooldown).

Run to execute this in your browser. Nothing is sent to a server.

The 400 path never reaches the second status. The timeout path prints two waits, then gives up. The circuit opens after three fails and skips the rest — even a later 200.

Retry the transport, not the business rule. A missing field is not flaky. A rate limit is.

How agents use this

Tools fail. Models time out. Rate limits hit. Your loop must retry the right errors, cap the tries, and back off. A circuit stops a loop of failing calls against a down API. Without this, one bad line becomes a hundred paid calls. Next you will overlap waits. First you must know which waits are worth repeating.

The model should see one observation: “search timed out after 3 tries.” It should not see three identical timeouts unless you want it to change query. Collapsing retries inside the tool adapter keeps max_steps honest.

Budget math should count failed tries if they hit a paid API. A 429 retry still cost a request. Your ledger is not only successful 200s. Print attempt numbers in the tool log. Then cost reviews make sense.

Honor Retry-After when a real server sends it. This box has no headers on the 429 body beyond an error string. On a laptop, if the header is a number of seconds, use that as the wait instead of your list, still capped. If the header is missing, use the list. Never wait minutes in a user-facing turn without telling the user. Print “would wait” even when you really sleep.