Function Calling

Function calling is a loop, not a miracle flag on an API:

1. You send messages plus a list of tool definitions
2. The model returns a normal answer or one or more tool_call objects: name + arguments
3. Your server runs the named function
4. You append a tool result (the observation)
5. You call the model again until it stops calling tools

That is the whole protocol. Frameworks add glue. They do not add magic. The agent loop is a client of this protocol. It must not skip step 3 and pretend step 4 happened. It must not skip step 2 and invent a name from a paragraph.

Vendors differ on wire format: some put calls as structured fields on the assistant message, some as XML, some as JSON in the text. Your wrapper normalizes to {name, args, id}. Internally you have one shape. The dispatcher never sees vendor XML.

Arguments will be wrong

Expect:

• Strings where you wanted integers ("17" vs 17)
• Missing fields
• Hallucinated tool names
• Valid JSON that is still nuts (job_id: -1)
• Extra keys
• Partial JSON while streaming

Coerce only the boring cases (digit strings to ints). Reject the rest with an error observation the model can use. Streaming APIs may emit partial argument JSON. Do not execute until the object is complete. A half-parsed refund is how you pass amount_cents: 40 instead of 4000.

Never treat the arguments blob as a program. Parse JSON. Then pass keyword args into a registered function. If parse fails, return invalid_json. Do not try to “fix” trailing commas by running a second parser that also accepts Python literals. That second parser is how you accept more than JSON.

Tool ids matter when several calls return out of order. Echo the id on the observation. The assembler must match result to call. If you drop ids, parallel search becomes a shuffled bag.

Unknown names fail closed

launch_nukes is not a tool because the model said it. Return unknown_tool. Do not search globals(). Do not import a module because the name contains a dot. The registry is the allowlist. Function calling without a registry is a shell.

A 20-line if name == ... or a dict of callables is enough. Auto-binding every Python function will bind helpers you forgot the day someone adds a utility. Register tools explicitly. One registry per product surface: support bot versus billing bot. Copy-paste of the full catalog into every agent is how a shell ships on Friday.

Observations go back as data

The result is a tool-role message, or the vendor’s equivalent. It is data. Cap it. Redact it. Do not promote it into the system prompt. Truncation is a later lesson. The protocol rule is: every executed call gets a result, even if the result is an error. Dropping errors is how the model retries a write it already ran.

If you run tools in parallel, you still append one observation per call. Do not smash them into one string. The next model turn needs to know which name produced which blob.

Classroom dispatch

The live box has one real function, get_job, and a registry with that name only. It coerces digit strings for job_id. It rejects unknown names. job_id: "17" is common and allowed here. launch_nukes is not created by wish. Missing jobs return not_found inside the result, which is different from unknown tool — the name was legal, the id was not.

Run to execute this in your browser. Nothing is sent to a server.

What printed: "17" becomes job 17 failed. Job 99 is not_found. launch_nukes is unknown_tool. Three observations, three lessons: coerce the boring case, structured miss, fail closed on names.

What goes wrong

Executing streamed partial JSON. Treating a prose sentence as a call because it contains parentheses. Binding os because auto-discovery scanned a package. Swallowing TypeError and returning an empty string, so the model thinks the tool succeeded. Running the same call twice because the observation was not appended. All of these are protocol bugs, not “the model is dumb.”

How to test the protocol

Fake the model as a list of calls. Assert dispatch output for coerce, not_found, unknown_tool, and bad types. Assert that an unknown name never invokes a function — monkeypatch the registry values if you must. Assert observations keep ids. You still do not need a vendor.

Normalize the wire, then dispatch once

Vendors disagree about where the name lives, whether arguments are a string or an object, and how parallel calls are grouped. Your wrapper’s job is to make that disagreement disappear before dispatch. Internally you want {id, name, args} with args already parsed. If parse fails, you never call the registry. You return invalid_json with a hint to resend a complete object. You do not run a second, looser parser that accepts Python literals, comments, or trailing commas. Looser parsers are how you accept more than JSON.

Streaming is the other wire mess. Partial argument text is not an object. Buffer until the vendor says the call is complete. Then parse. Then validate. Then dispatch. If you dispatch on the first closing brace you saw, you may have truncated a number. Refunds of 40 instead of 4000 start there.

Every executed call needs an observation, including errors and unknown names. Dropping a result because it was ugly is how the model retries a write. Append the blob. Pack it. The next model turn is a client of that transcript, not of RAM on the worker.

Tool ids are how parallel results find their call. If the vendor omits ids, mint them in the wrapper and keep the map for the turn. Do not rely on array order after an async gather. Order is a rumor.

How agents use this

Normalize vendor payloads to one internal call object. Dispatch through one function. Append one observation per call. Stop when the model answers or the budget hits. The loop never talks to Stripe except through dispatch. If you cannot find dispatch in the codebase, you do not have function calling. You have a chat app with extra JSON.

Note:Partial JSON is not arguments. Wait for the end of the call object.