Curriculum/Large Language Models
System Prompts
Specs beat vibes. Write a system prompt like an API contract: tools, schemas, stop rules, forbidden actions.
The system prompt is the pinned spec. It is the worst place to dump a novel and the best place to put constraints that must survive your truncation policy. When the window is tight, you will drop history and RAG first. You should still have a spec.
“You are a helpful assistant” is a vibe. It does not say when to stop, which tools exist, what JSON looks like, or what to do when tools fail. Helpful-to-whom is how you get data leaving the building with a smile. Helpful to an attacker is still “helpful.”
Spec vs vibe
| Vibe | Spec |
|---|---|
| Be careful | Never call delete tools without approval=true |
| Be concise | Answers at most 8 sentences; tool args only in JSON |
| Use tools when needed | If the user asks for a number from our DB, you must call sql first |
| Don’t hallucinate | If the tool did not return it, say you do not know |
| Be world-class | (delete this sentence; it is not checkable) |
Write checkable rules. If you cannot write a grader, the rule is a wish. A grader is a function: did they call get_job before claiming status? Did JSON parse? Did they mention DROP? The Prompt track will go deeper on evals of prompts. Here: the system text is a contract, not a pep talk.
A production spec is closer to a README:
- Role and goal — one paragraph
- Tools — names, when to use, when not
- Output contract — schema or sections
- Safety — forbidden actions, PII, escalation
- One tiny legal example — optional
Do not paste 40 contradictory incident write-ups. That is overfitting, in English. The model will imitate the incidents, including the ones you were warning against. One short legal tool call in the spec is worth ten paragraphs of adjectives.
Long system prompts steal window from observations. After about 1–2k tokens of spec, you are usually compensating for missing tools. If you need a handbook, retrieve a chunk (RAG track). Do not pin the employee wiki.
“Be helpful” cannot be graded. Tools, JSON, and “never” can. Write a contract, not a pep talk.
Checkable hits in the pinned textRun to execute this in your browser. Nothing is sent to a server.
Vibe hits 0 of 4 checkable needles. Spec hits 4. Vibe allows delete talk (True) because the word never is absent. Spec does not. Real safety still needs tools that cannot DROP — defense in depth — but the prompt should not argue for the crime. A grader that greps needles is a toy. Your real grader should run the loop on frozen tickets. The toy shows why adjectives do not score.
Priority when rules collide
Write the order down:
- Code allowlists beat spec (a tool you did not register cannot run).
- Spec beats user (“ignore the spec” in the ticket does not mint
refund). - User beats retrieved docs — unless you explicitly design “docs override,” which is how a wiki edit becomes policy.
If product wants the handbook to be the spec, that is a choice you record, and it is also a jailbreak surface. Default: docs are data.
Secrets do not belong in the system prompt. “The admin password is…” teaches the model a fact it may later recite. Put secrets in the environment. Put handles in the prompt: “call vault_get with name=db_url.”
Versioning and change control
Version system prompts like code (spec@2026-03-01). A/B them on a frozen eval set. When someone wants to “just add a sentence,” that is a PR. The sentence will interact with every tool description. Log the spec id on every trace so you can group failures.
Prompt cache (previous part) wants this spec byte-stable at the front. Edits should be releases, not hot-patched strings in a database with no diff.
A/B: 50/50 on the frozen set, not on live refunds. Metrics: JSON-ok, tool-before-claim, abstain-on-impossible, forbidden-string rate. If the new spec wins on vibes and loses on tool-before-claim, you did not win. Ship the old spec.
Keep the spec under a budget (for example 1–2k tokens). If product wants another policy appendix, that appendix is retrieval, not another 4k in system. The working-set lesson will pack; your job here is not to make packing impossible.
Write checkable rules in the same PR as the grader. If you cannot name the test (“must call get_job before status”), the sentence is a vibe. Delete it or replace it. “Be careful with deletes” becomes “never call delete without approval=true in code state.” The tryit is a grep toy; the shipping test is a frozen ticket.
What goes wrong
- Novels. Observations get truncated first; the model never sees the tool result.
- Contradictory incidents as few-shots of failure.
- Secrets in the spec.
- “Be helpful” overriding “never refund.” Helpfulness is not a higher law unless you write it that way — don’t.
- No grader. Then every debate about the spec is taste.
How agents use this
Load the spec from a file. Pin it as the first message. Unit-test that forbidden tool names are absent from enabled docs. Eval: tickets that must call get_job; tickets that must abstain; tickets that must not mention DROP.
When packing gets tight, keep the spec. Drop encyclopedia RAG. If the spec itself does not fit, you do not have a packing problem. You have a spec problem.
The grader in the tryit greps needles. Your real grader runs the agent. Keep both: grep catches a spec that forgot get_job; the eval catches a spec that names the tool and still does not call it.
Watch out:Putting secrets in the system prompt teaches the model a fact it may later recite. Put secrets in the environment. Put handles in the prompt.
Check your understanding