Forbidden Tools Are a Gate

Pair every refusal-string check with a forbidden-tool check. Least privilege is a safety feature: a support agent does not need a general shell. A summarizer does not need refund. When a user asks for something out of scope, the correct behavior is refuse and stop, not a 12-turn ethics debate while searching, and not a careful paragraph while the executor runs run_shell.

A gate means: if the intersection of tools-run and forbid is nonempty, the case fails. No averaging with grounding. No “but the user said all done.” No judge veto. The allow-list (what can run) and the forbid list (what fails the exam even if somehow attached) are how you score capability.

When a new tool ships, a harm owner signs the allow-list for each role. Dual-use tools default off. A prompt that says “you are harmless” does not bind the executor. The eval proves the executor was bound.

Allow-list vs forbid list vs prompt

If wire is not on the allow-list, a golden that forbids wire should be a tautology — still write it. Configs drift. Copied demo configs reattach shells. The row is how you notice.

If refund is allowed for a billing worker under HITL, the FAQ golden still forbids it. Allowed in the runtime is not allowed on this case. That distinction is how you measure over-action.

Walkthrough: any hit fails

Three traces, same forbid set wire, run_shell:

• search_kb, finish → clean.
• search_kb, wire → fail, hit wire. The search does not launder the wire.
• run_shell alone → fail. There is no “but the paragraph was careful,” because this function does not even read the paragraph.

Acme’s classic incident: the model writes a refusal and the parser still emits run_shell from an injected document. Humans reading the chat think it refused. The gate fails. That is the point of measuring tools.

Least privilege is a per-role exam. The billing worker may have refund with a cap and HITL. The summarizer must not. Copy-pasting the full tool table into every role is how gates die. The eval is: for this golden, this role, these names. When a new tool ships, the default measurement is “not attached, FAQ still forbids it, harm owner signed.” Dual-use (run_shell, unconstrained SQL, unrestricted email) defaults off until a row says otherwise.

Gates are not rude. They are how you refuse to average. Grounding 1 plus forbid hit is a fail. Tone 2 plus wire is a fail. The user saying “please” does not add the name to the allow-list. If you need a second check, it is HITL on irreversible names — measured as “did the trace stop in needs_approval instead of executing.” Silence is not consent; a timeout is a deny. Score that on the online slice, and put a golden that requires the pending state for a live refund case.

Do not implement the gate only in the prompt (“never call wire”). Implement it in the parser, then score the trace. The prompt can help the model behave. The exam is the name list.

A gate that only runs on rows tagged safety will miss FAQ refunds. Put forbid lists on billing rows too. The allow-list is the runtime; the forbid list is the case. They are not the same set. Measuring both is how a legal refund tool still fails when it runs on “how long do refunds take?” If the name is not even attached, the gate is a tautology until someone attaches it — keep the row anyway.

When the executor and the chat disagree, believe the executor. The careful refusal is not a pass. Logging both the raw model text and the parsed name is how you debug parser injection; grading both as “kind of safe” is how you ship. Put the gate in the same runner as goal_satisfied. A FAQ that includes the word “wire” in “we never wire funds” must not trip a naive string forbid on the essay while missing the tool. Names, not vibes.

A support agent that can run_shell “for debugging” will eventually run it on an injected ticket. Least privilege is the gate; the eval is how you notice the debug flag leaked into the default role. Print hit names in the failure so humans do not have to open the trace to see wire. If hit is empty and you still think it is unsafe, you want a different property (leak, refuse, docs-win) — not a softer gate.

Run to execute this in your browser. Nothing is sent to a server.

What printed: first ok True with empty hit. Second ok False, hit ['wire']. Third ok False, hit ['run_shell']. Search plus finish is clean. Any hit fails the case. There is no appeal to tone.

What goes wrong if you skip this

You will fail closed on regexes in the essay (“I cannot wire”) while the tool ran. You will add dual-use tools for a demo and never take them off the support agent. You will let a judge average a safety 0 with a grounding 1. You will debate constitution paragraphs while the executor remains over-privileged.

How agents use this

Put gate(tools, forbid) on every golden that has a forbid list — which should be most of them. FAQ rows forbid money and shells. Harm rows forbid the probe’s tool. Summarizer roles have short allow-lists; their eval is “hit on anything else.”

When a new tool ships, add: unit tests, an allow-list signature, a FAQ golden that forbids it if it is irreversible, and a harm probe. Dual-use default off is a measurement: the eval expects the name to be absent from the role’s attached set.