Encoder vs Decoder

There are two classic transformer shapes.

An encoder lets every token see every other token (bidirectional attention). BERT-style. Good at understanding a whole sentence: classify, embed, fill a blank in the middle.

A decoder lets token i see only tokens ≤ i (causal mask). GPT-style. Good at generating the next token, then the next.

An encoder-decoder (old translation models) encodes the source with a bidirectional stack, then a decoder attends to that memory while generating the target. Cross-attention sits in the decoder: queries from the target so far, keys and values from the source encoding.

Most stacks you will call for an agent loop are decoder-only: everything lives in one sequence. Docs, tools, and instructions are concatenated into the prompt. There is no second encoder behind the curtain. Retrieval is not a hidden encoder. Retrieval is more tokens you pasted.

A wrong picture

A wrong picture is: “encoder means it is smarter.” Encoder means bidirectional mix. That is the right inductive bias for “what is this sentence about?” It is the wrong bias for “write the next id without cheating.”

Another wrong picture is: “my chat model has a secret encoder for documents.” If the product is decoder-only, your documents are in the same causal sequence. They compete for softmax mass. They get positions. They can be truncated.

A third wrong picture is: “you can stuff chat completions into an embedding endpoint.” Embedding models are often encoder-like (or a decoder with pooling). Chat models are decoders trained to continue roles. The vectors are not interchangeable. Mixing those spaces is the embedding-table lesson again.

Masks are the difference you can draw

For a sequence of length 4, query rows vs key columns:

Encoder: a full square of allowed. Position 0 may look at 3. Position 3 may look at 0.

Decoder: a triangle. Position 0 looks only at 0. Position 3 looks at 0, 1, 2, 3. The future is dots (blocked).

That triangle is next-token training. If the decoder could see the future token, “predict the next id” would be cheating. Teacher forcing still feeds the true past (next lesson on pretraining), but never the future id being predicted.

Fill-in-the-middle and masked-language models exist too. A masked model hides random tiles and guesses them from both sides. Chat agents still wrap a causal decoder almost every time.

Prefix language models (mask the prompt as bidirectional, generate the answer causal) show up in research. Your hosted chat still looks like a transcript with a causal generator.

Print a Y/. mask for both kinds

No neural net. Nested loops. Y means allowed, . means blocked.

Run to execute this in your browser. Nothing is sent to a server.

Encoder: a full square of Y. Decoder: a triangle. That picture is the lesson. Cross-attention would be a second grid: every decoder position (that exists so far) against every encoder position. Decoder-only skips that second grid and puts the “source” into the same triangle as extra past tokens.

If you add a pad mask later, some Y cells become blocked too. Causal plus pad is the production decoder mask. Bidirectional plus pad is the production encoder mask.

What each shape is for

Agents that generate actions need a decoder. Agents that index memory often need an encoder-like embedder. Those are two models, two tokenizers, two vector spaces. Do not mix them.

Decoder-only RAG is: embed with the embedder (encoder-like), retrieve strings, paste strings into the decoder prompt. The decoder never saw the embedding table of the embedder. It sees tokens.

Common mistakes

Calling a decoder a “BERT” because both are transformers. The mask is the product difference. Bidirectional mix vs causal mix. Wrong mask, wrong job.

Pasting retrieved chunks into an embedding model and expecting a tool call out. Embedders are not generators. Generators are not drop-in embedders. Two models, two tokenizers, two spaces.

Hoping a chat decoder will “read both sides” of a JSON key you put after the slot being predicted. Causal means the key after the cursor is the future. Put the schema before the object you want filled, or use a grammar. Do not hide the spec on the right.

Building a toy encoder-decoder and then concatenating source and target into one causal stack “to simplify.” You just built a decoder-only model. That can be fine. It is not the translation picture. Name it honestly.

How agents use this

When a product sells an embedding model, it is often encoder-like (or a decoder with pooling). When it sells a chat model, it is a decoder. Do not stuff chat continuations into an embedding endpoint or the other way around. For retrieval plus generation, embed with the encoder-like model, generate with the decoder, and never mix vector spaces.

You will not switch a hosted chat model into bidirectional mode to “make it read the middle better.” Packing the window (later) is the lever. Architecture is frozen.

If you train a small classifier on tickets, an encoder is a natural backbone. If you train a small policy that emits tool names left to right, a decoder is the backbone. Same block pieces, different mask.

• Chat / tools: decoder, causal triangle.
• Embeddings: encoder-like, full square, then pool.
• No secret encoder: pasted docs are more decoder tokens.
• Do not mix spaces: embedder rows are not chat rows.
• Honesty: causal mask is why next-token training works.

Note:Prefix language models (mask the prompt as bidirectional, generate the answer causal) show up in research. Your hosted chat still looks like a transcript.