What Is an LLM?

A large language model (LLM) is a program that guesses the next token. A token is a chunk of text — a word, a piece of a word, a punctuation mark, or a bit of code. The Transformers track already showed how tokens, attention, and decoding work inside the network. This track is the product around that network: the HTTP call, the bill, the message list, and the wrap you put on the guess so an agent can ship.

When the model is big enough, those guesses look like skill. It can follow instructions, draft code, extract a field, and imitate an expert. That look is useful. It is also the trap. Fluency is not a proof. The training game is plausible continuation, not “look this up and only say it if it is true.”

“Large” is not a legal word. It means: too big to train on your laptop, usually served through an API, with a tokenizer, a context window, and a decoding policy. An API is a contract: you send data, you get data back. The context window is the maximum number of tokens the model can read in one call. The decoding policy is how the server turns a pile of next-token scores into actual text (greedy, sampled, stopped at a limit). The product you buy is not intelligence. It is tokens in, tokens out, with a wait time and a price.

The stack you actually touch

You never “talk to the weights” in production. You talk to a stack. Name the layers or you will debug the wrong one.

Joeven agents live in the last row. If you confuse the API with the agent, you will patch the prompt when the stop rule is missing, or blame “the model” when you never appended the tool result. The model is a function. The agent is everything that calls that function more than once.

The weights do not update when a user talks to your product. Each call is a fresh read of the messages you send. Memory is your object: the transcript, the database, the files. Saying “the LLM remembered” means you resent old text.

Good at / bad at

Good at: turning messy language into a draft of structure; proposing the next tool from a documented list; writing glue code; extracting a field when the pattern is sitting in the prompt; summarizing a page you actually provided; sounding like a careful employee.

Bad at, alone: knowing if a fact is true today; exact arithmetic on long numbers; remembering anything not in the weights or the window; obeying a policy it can also be talked out of; being the system of record (the place the business treats as truth); counting, totals, and checksums you need to guarantee.

Stable facts and small math sometimes work because they were easy patterns in pretraining. “Paris” after “The capital of France is” is a dense pattern. “The CEO of Acme as of today” is a retrieval problem wearing a language costume. Retrieval means: go get a document or a row, then speak. Do not skip the get.

Run to execute this in your browser. Nothing is sent to a server.

The box is a lookup table pretending to be a model. The first two prefixes hit dense patterns. The third returns a plausible name that is not checked against any database. That is the honest demo: continuation, not knowledge. A real model is a huge, fuzzy version of the same game. Your wrap — schemas, tools, evals, budgets — is how you stop shipping the plausible name as if it were HR.

Change a prefix so it is missing from table. You get "...". A hosted model will not print dots. It will invent a sentence. That difference is why agents need a check after the guess, not a vibe in the prompt that says “be accurate.”

A walkthrough: Maya’s CEO question

Maya builds a briefing agent. A user asks, “Who is the CEO of Acme today?” She sends the question to a chat API. The model answers with a confident name and a biography. The name is last year’s CEO. The biography mixes two people. The UI shows a green check because the JSON parsed.

Nothing in the stack was “broken.” The model did the job it was trained for: continue the question as if it were a news blurb. The product was broken because Maya treated the completion as a database. The fix is not a bigger model. The fix is: call a search or an internal directory tool, put the result in a tool message, and refuse to name a CEO if the tool is empty. That pattern — guess only after an observation — is the rest of this track and the tools/RAG tracks after it.

What an LLM is not

It is not a knowledge graph with proofs. It is not a Python interpreter (do not exec model text). It is not your authorization layer. It is not a clock; “today” in the prompt is a string you injected, or it is a guess. It is not cheaper than a SELECT when you already have the row.

Teams skip this list because the demo is pretty. Then finance asks why the bill is a novel, legal asks why the bot invented a policy clause, and ops asks why the agent refunded twice. All three failures start by treating tokens-out as truth.

What goes wrong

• Using the model as the system of record. Ticket state lives in the database. The model may describe it after a tool call.
• Skipping tools for “easy” facts. Easy facts go stale. Today’s hours, prices, and names are tools or retrieval.
• Debugging the weights first. Check the message list, the finish reason, and the parser before you switch vendors.
• Paying for a poem when a table would do. The last lesson of this track is when not to call a model. Start that habit here.
• Confusing chat UIs with APIs. Pasting customer data into a consumer chatbot is a leak, not an integration.
• Ignoring the window. If the spec, the tools, and the observation do not fit, the model did not “forget.” You overflowed.

How agents use this

Treat the LLM as a guessing policy over tokens. Wrap it until the wrap is safe enough to ship. The wrap is not a paragraph of adjectives. It is code:

• Schema: the model may only emit actions you can parse.
• Tools: facts and side effects go through functions you wrote.
• Evals: a frozen set of tickets that must pass after you change the spec.
• Budgets: max steps and max dollars, then stop.
• Logs: model name, usage, finish reason, message roles.

If you can replace the model with a lookup table or a workflow, do that. Save the model for branches you cannot draw: messy language, a long menu of tools, a draft the user will edit. A branch you can draw belongs in if and SQL.

Name the function complete(messages) (or chat) and keep it tiny. The agent loop should not know whether the other side is a hosted API, a local server, or a fake client in a test. Tests should not need a network. Joeven’s Try it boxes cannot hit a network on purpose. That is the same discipline as a unit test: script the assistant turn, assert the next tool.

Tip:The rest of this track is the wrap — APIs, money, roles, JSON, knobs, lies, packing, and model choice. Attention math already had its track. Prompt injection as a writing craft is the next track. Here you learn the call.