Secrets and Security
Secrets live in a manager or the tool service, never in prompts, repos, or traces. Treat the model as an untrusted client; redact before the scratchpad; pin dependencies.
Agents are secret amplifiers. They log more, they retrieve more, they send more text to a third-party model API. Your threat model is not only “hacker on the laptop.” It is also: prompt injection, a curious employee in the trace UI, a vendor breach, and a model that echoes a key it saw in a tool dump.
Where secrets live: a secret manager or the environment of the tool service, not the prompt, not the repo, not the trace. Short-lived credentials where possible. Separate keys per environment (dev / stage / prod) with hard spend limits on experimental keys. If a tool result might contain a key (config files, error pages), redact before it enters the scratchpad.
Never paste secrets into Joeven, tickets, or screenshots. Never dump the whole environment into an observation. Pin tool dependencies. An agent that installs a package named by the model is a remote-code product. Computer-use on an unsandboxed desktop is the same class of bug.
How the box actually works
| Place | Allowed to hold Stripe? | Notes |
|---|---|---|
| Prompt / template | No | The model will quote it |
| Scratchpad / observations | No | Redact first |
| Trace store / exporters | No | Subprocessor + leak |
| Worker env | No (confused deputy) | Model key only, scoped |
| Tool service env / secret manager | Yes, scoped | Short-lived if you can |
| Repo / CI logs | No | Scan; rotate if it landed |
| Trace UI | Masked | Reveal audited |
Keys live in the manager or the tool service. Redact before the scratchpad.
Secrets stay off the promptThe model host cannot reach the database except through tools. Tool authz uses the signed job tenant (next lesson is the filter; this lesson is the secret and the echo). Rotate on a schedule and after any suspected leak.
The trace UI is a privileged app: SSO groups, audit logs on “view trace,” no world-readable links. Security reviews read tool authz and redaction, not only the system prompt’s manners.
Owners: security owns rotation policy and secret manager. Domain tools own scoped roles. Runtime owns “do not print env.” Platform owns pinning installs and sandboxing computer-use.
Short-lived credentials: a worker that fetches a 15-minute Stripe-scoped token from the manager at slice start is better than a long-lived key in an env file baked into the image. Separate spend limits on stage keys so a leaked demo key cannot become a production-sized bill. Prod keys never live in staging.
The model provider is another place secrets must not go. If a 500 body with a key reaches complete(), you have exported the secret to a third party even if your trace store is clean. Redact before the assembler, full stop.
An echo ticket
A billing 500 page included key=sk-live-.... The tool returned the HTML. The next model call proposed use sk-live-... please as a “fix.” That completion landed in traces and in a vendor exporter. Rotation, redaction, structured errors. The golden was: after get_invoice, stored observations match REDACTED not sk-, and cross-tenant args still PERMISSION_DENIED.
A second finding in the same review: the worker image could pip install a name from JSON. Disabled. The model is not your supply chain.
Run to execute this in your browser. Nothing is sent to a server.
Own tenant: ok, with a redacted note — you should see [REDACTED_KEY], not the demo key. Other tenant: denied, no note, no key. The prompt line is already redacted so a jailbreak-style instruction cannot carry the live secret into the next complete(). Own tenant, redacted error. Other tenant, denied. The prompt never received the live key. Concatenation built the fake error on purpose so we never use an f-string.
What goes wrong
Keys in prompts “so the model can call Stripe.” Keys in few-shots. Keys in CI artifacts. One key for stage and prod. Unlimited experimental keys. Logging args without redaction. Model-named packages. Unsandboxed desktop control. Trace UI shared with the whole company.
Treating injection as a prompt wording problem only. Injection is how untrusted text reaches a tool. Isolation and redaction are the controls; wording is extra.
A leaked stage key with prod-sized spend limits is a prod leak. A screenshot in a ticket system with looser ACL than the trace store is a side channel. “We only use test keys in CI” that are actually live is a scanner finding, not a debate.
How to test it
- Redact fixtures as in tracing.
- Worker image config: Stripe not in worker env.
get_invoicedeny + redact tests.- CI secret scan on the repo and on sample traces.
- Computer-use / install-from-JSON disabled in prod flags.
- Rotation drill: time to revoke and replace.
Do not test “the prompt says not to reveal secrets.” Test that the secret never entered the prompt.
How agents use this
Ban running model-generated code on boxes with secrets. Pin dependencies. Redact before scratchpad. Rotate on leak and on a calendar. Put spend limits on every vendor key so a leak cannot also be an infinite bill.
When you add a tool, ask where its key lives and what a 500 body looks like. If you do not know the 500 body, you have not redacted it.
SSO on the trace UI is part of the agent product. Curious employees are in the threat model. Audit reveals.
Pin lockfiles. Disable install-from-JSON in prod flags. Computer-use stays off until the sandbox is a real box with no secrets, not a developer laptop. Injection is how untrusted text reaches a tool — isolation and redaction are the controls; a polite constitution is extra.
Check your understanding