Base vs Chat Models

A base model is the pretrained continuer. Give it a webpage, it writes more webpage. Give it Python, it writes more Python. Give it a half-finished email, it finishes the email in the same voice. It does not wait politely for a question. It does not know it is an “assistant.” It knows how to keep going.

A chat (instruction-tuned) model was trained further to answer in roles: system, user, assistant. That is why it waits for you. It is also why it hedges, refuses, or moralizes in ways the base model would not. The extra training is not a different kind of attention. It is more next-token training on a new kind of document: conversations with a spec at the top.

When a blog says “this model can do X,” check whether they meant base, chat, or a tool-using wrapper. Those are three different products on related weights. A leaderboard number for “GPT can do X” is often a chat model plus tools plus a hidden scaffold. Your hosted call may be weaker, safer, or both.

Three trainings, one family of weights

1. Pretraining — continue the internet (and code, and books). This is the base model.
2. Instruction / chat tuning — continue conversations that look like “system / user / assistant,” including “helpful” answers and some refusals.
3. Preference tuning (RLHF, DPO, and cousins) — prefer answers raters liked, which includes “don’t help with this class of request.”

You do not need the math of those methods in this lesson. You need the product fact: chat behavior is extra training plus a template. The template is a way of wrapping roles in special tokens so the model can see who spoke. The Transformers track already showed why special tokens matter. If you ignore the template and paste a raw document, you are feeding a different string than the vendor used in training.

Almost every commercial chat API hides the template. You send JSON messages. The server wraps them. That is a gift. It is also why “I copied a completion prompt from a 2022 blog into Chat Completions” fights the model: you are asking a chat model to pretend it is still a base model.

What you send vs what the model sees

You send a list of dicts. The vendor turns that list into one token sequence with markers for roles. Tool calls may be extra structured fields, not prose. Your agent should think in messages. Do not try to reverse-engineer the wrapper unless you are hosting open weights and you own the tokenizer files.

Base-style hosts still exist: some research endpoints, some self-hosted completions, some “fill in the middle” code models. For agents, default to chat. Completions are a special case you opt into when the host has nothing else.

Run to execute this in your browser. Nothing is sent to a server.

The base function continues the policy paragraph. The chat function answers the user and, in this toy, even proposes a tool. Same family of idea, different default. A real chat model might still continue a document if you force it, but you will be fighting later training. Use the chat template. Put the spec in system, the question in user.

Context windows, tools, and JSON mode are product features

A longer window, native tool-calling, and “JSON mode” sit around the weights. They change what you can parse and how much history you can afford. They do not turn the network into a database. A 128k window filled with a stale wiki is still stale. JSON mode that emits {"ceo": "plausible"} is still a guess.

Tool-calling is a chat feature: the assistant turn may contain a structured call instead of (or as well as) prose. You will wire that in later lessons. The point here is: do not expect a base model to emit a clean tool call just because you wrote a poem about functions. Chat models were shown that shape. Base models were shown websites.

Refusals and hedges are trained, not magic ethics

Preference-tuning is why chat models refuse some requests — and why they sometimes refuse too much (a medical-adjacent question that is actually a billing FAQ) or too little (a polite jailbreak; the Prompt track covers injection as text). Your runtime must still allowlist tools. A refusal in prose is not an authorization layer. A smiling “sure, I’ll refund” is not a payment API.

If you need a model that will discuss security incidents, pick a vendor and a spec that allow that topic, plus human review. Do not keep a base model in production “because it won’t refuse.” It also won’t follow your stop rules.

What goes wrong

• Pasting “continue this document” into a chat model and wondering why it answers as an assistant.
• Assuming a chat model will follow a safety poem a base model never saw. Put refusals in code too.
• Reading a base-model paper and buying a chat API, then being surprised by hedges.
• Using one global temperature for both JSON tools and marketing copy. Knobs come later; the split starts with “this is a chat product.”
• Pretending roles are optional. Empty system + a huge user blob is how specs disappear under tickets.

How agents use this

Call the chat API unless you have a rare completion-only host. Put the spec in system, the human in user, and append exactly what the assistant produced (text and/or tool calls). Never invent a fake assistant confession to “steer” the model unless you are writing a test double and you label it as one.

Keep a tiny adapter: complete(messages) -> assistant message + usage + finish_reason. Behind it, hosted chat, local chat, or FakeChatClient. The loop does not change.

When you host open weights, load that model’s chat template. Mixing Model A’s roles with Model B’s tokenizer is a silent quality bug. Tokenizers differ; prompts that depend on exact token counts need a second look after a swap. That constraint beats a screenshot of a leaderboard.

Note:“This model can do X” on a leaderboard is often a chat model with tools. Your hosted call may be weaker, safer, or both. Measure your tickets.