What Is Not a Tool

A tool is not:

• A sentence in the system prompt that describes an API
• A fake “I have access to the internet” persona
• A comment in the UI
• Dumping a 40-page SDK into context and hoping the model types a curl you then run as a string
• A Slack emoji, a sparkle icon, or a marketing bullet

If your runtime cannot validate arguments and execute it, it is not a tool. The model can still talk about it. Talking is not calling.

This rule is how you keep the catalog honest. Prompting tracks already told you not to advertise missing tools. Here it is a runtime rule: the list you send to the model is generated from the same registry the dispatcher uses. If a name is missing from the dict, it is missing from the prompt. If someone comments out a handler on Friday, the name disappears from Monday’s sessions.

Prompt text is not a handler

“You may browse the web” does not open a socket. “You may refund up to $50” does not cap Stripe. Those sentences can still help the model choose. They cannot enforce. Enforcement is schema, allowlists, and credentials. If you only have the sentence, you have theater.

SDK dumps fail in a second way. The model types a plausible curl. A junior wrapper pipes it to a shell. Now you have a tool you never designed: unrestricted command execution, named “the model was being helpful.” Register functions explicitly. Parse JSON. Call a Python function. Do not treat model text as a program.

UI copy fails in a third way. A button labeled “AI can file tickets” trains users. If create_ticket is not in the registry, the button is a bug. Product and runtime must share the catalog.

Sync vs wait

Synchronous tools return a small JSON blob now (get_job). The loop can append the observation and call the model again in the same slice.

Asynchronous tools return a handle (job_id) because compile, email, or a human will take seconds to hours. Agents must poll or wait on a queue. If you pretend a 10-minute refund is sync, you will timeout and double-submit.

A handle is still a tool result. It is not a promise that the work finished. The next turn calls poll(job_id). It does not invent “compile finished.” If poll says running, the loop should wait or stop, not start a second compile with a new id.

Long work belongs in a worker you already trust: a CI system, a mail provider, an approval queue. The tool starts it and returns an id. The tool does not block the Python process on sleep until the universe is done. Blocking is how one hung job freezes the agent.

Registry is the only advertisement

The live box has a prompt claim and a runtime dict. PROMPTED says the assistant may browse. RUNTIME only has get_job. is_real_tool("browse_web") is false. That is the correct product state: the prompt lied, the registry did not, and only the registry counts. In a real system you would fix the prompt by generating it from RUNTIME, not by teaching the model to ignore the lie.

Jobs show the async shape. start_compile returns j1 still running. poll("j1") is still running. poll("j2") is done. The model does not get to skip poll because it is impatient. Impatience is a retry incident.

Run to execute this in your browser. Nothing is sent to a server.

What printed: the prompt claims internet; browse is not in the runtime; get_job is. Start returns a running id. Polling j1 is still running. Polling j2 is done. Unknown ids are not_found, not “probably fine.” Advertise poll. Do not advertise browse.

What else people mistake for tools

Retrieval is a tool when it is search(q) with a schema. A silent pre-fetch that pastes ten chunks into the system prompt is not a tool call. It is assembler work. The model did not choose it. You cannot log it as a call the model made. You can still log it as context. Do not confuse those logs.

Memory writes are tools if the agent chooses remember(note). A hidden summary job after every turn is runtime. Both are fine. Names matter when you grade “did it call the right tool?”

MCP servers expose tools. MCP is a transport, not a brain and not a permission system. You will get two lessons on that. For now: a server that is not installed is not a tool, even if a blog post named it.

How to test “is this a tool?”

Write is_real_tool(name) against the registry. For every name in the prompt fixture, assert true. For a name the marketing site loves, assert false until the handler exists. For async tools, assert that the start result contains a handle and that a second function polls it. If start returns a 10 MB log and no id, you built a sync trap.

How agents use this

Advertise in the prompt only names that exist in the registry. Long jobs return ids. The next turn calls poll(job_id) — it does not invent “compile finished.” The loop is a client: it must understand running versus done as data, not as vibes.

When a vendor ships a new “computer use” demo, ask: what is the function signature, what is the allowlist, what is the log line? If those are missing, you watched a video, not a tool. A checkbox in an admin UI that says “enable browsing” is still not a tool until browse_web exists in the registry and the worker can actually fetch. Toggle the flag only after the handler, schema, and fixtures exist. Marketing copy that lists twenty integrations is a roadmap. The registry is the product.

Note:Disabled tools must vanish from both prompt and dispatcher. A 404 from the dispatcher after you advertised the name is how models invent args for ghosts.