Hidden Scratchpads

Shown CoT is in the user-visible reply. Good for tutoring. Bad if the scratchwork contains private tool output. Bad if a lawyer will read the scratchwork as if it were a proof.

Hidden CoT (a scratchpad field, or a vendor “reasoning” channel the UI never prints) is for the system: the loop may read it; the human sees only the final contract.

Treat hidden traces as logs:

• Retain them for evals and incident review
• Redact them for customers, support exports, and screenshots
• Never trust them as ground truth
• Never put secrets in the spec just because the scratchpad is “hidden”
• Never assume hidden means encrypted — it is still text in a vendor payload and in your store

A third option is no free-form thought: only JSON fields. That is usually what you want in an agent. thought as a short string you cap is a compromise. A novel in thought is a cost center and a leak.

Hidden is not a safety control

Users will ask the model to “think about ignoring the spec.” Jailbreaks still work on hidden channels. Secrets in tool results still land in scratch if you put the raw result in the window. Hidden only means the default UI does not print it. Support will still paste it. Logs will still hold it. A misconfigured debug flag will still show it.

Vendors that bill “reasoning tokens” separately: cap them. A 4k silent monologue is a product choice. Measure it. Put it on the same spend dashboard as visible tokens.

If the vendor forbids showing hidden reasoning, do not show it. If your product wants a trail, generate a user-safe trail from the contract and from allowlisted facts, not by forwarding the raw scratch.

What the user should see

The user sees the output contract: status, answer, maybe sources. They do not see API keys from a tool error. They do not see “I considered wiring money.” They do not see another tenant’s id that appeared in a debug field.

The log sees a redacted scratch: tool names, error types, step counts. Redaction is a pass over the string, not a hope that “hidden” did the work.

Grade both story and answer when the user will see steps (tutoring, audit). Grade the contract always. A pretty trail that contradicts the sources is a fail if you displayed the trail.

Run to execute this in your browser. Nothing is sent to a server.

What printed: the user object has status and answer, no key. The log line has [redacted]. The raw scratch still has the key until you redact storage too. Hidden is not encrypted. If you persist turn["scratch"] unredacted, you stored a secret.

Walkthrough: the interesting debug flag

A developer enables “show reasoning” on the support UI because the trail is fascinating. Job 17’s vendor error included an API key. The customer screenshot hits Twitter. Hidden was never a vault; it was a default CSS rule. The fix is: UI always renders the contract; traces redact on write; debug flags still run the redaction pass; secrets never belong in the spec “because scratch is hidden.”

Vendors that bill silent reasoning tokens will happily generate 4k of monologue. Cap them like any other completion. Put the cap next to max_steps.

What goes wrong if you skip this

You leak tool output. You treat scratch as a proof in a lawsuit. You store keys. You pay for novels. Jailbreaks still work; you just cannot see them in the customer bubble. Logs without redaction are the same incident with extra steps.

Shown CoT is user-visible. Hidden CoT is a log. Hidden is not encrypted, not a safety control, and not a proof. UI renders the contract. Traces redact on write. Vendor reasoning tokens get a cap like any completion. If the product wants a trail, generate a user-safe trail from allowlisted facts, not by forwarding raw scratch.

A third option is no free-form thought: only JSON fields. That is the usual agent default. thought as a short capped string is a compromise. A novel in thought is a cost center and a leak.

Common mistakes

If a lawyer will read the trail, you are in shown-CoT land and you must grade claims against sources. If a customer will screenshot the UI, you are in contract-only land. There is no third product that “shows a little scratch.” A little scratch is how keys travel. Redaction belongs in the write path of the trace store, not in a hope that nobody will click Debug.

How agents use this

UI: final object only. Trace: scratch redacted. Eval: sometimes the story is wrong and the answer is right (or the reverse). Grade both when the user will see steps. Grade the contract always. Do not print hidden reasoning to end users just because it is interesting. Hidden is a default hide, not a vault. Redact on write. Cap silent reasoning tokens. If a debug flag can show raw scratch, the flag must still run redaction or it is a leak with extra clicks. Interesting is not a reason to show a log. The contract is.

Watch out:Do not print hidden reasoning to end users just because it is interesting. It can contain secrets from tools and it is not a proof.