Matrices

A matrix is a table of numbers. In Python we use a list of rows, each row a list of equal length. Shape is (rows, cols). A matrix W with 2 rows and 3 columns is 2 by 3.

You add matrices of the same shape, slot by slot. You scale a matrix like a vector. The operation that earns the data structure is matrix-vector multiply: it sends a vector of length cols to a vector of length rows.

For each row of W, take the dot product of that row with x. The results become the components of y. If W is m by n and x has length n, then y has length m. Shape errors are len(row) != len(x). Raise them loudly.

A wrong picture

A wrong picture is: “a matrix is a spreadsheet I read as text” or “I can multiply any two tables.” Shape is a contract. 2 by 3 times a length-2 vector is illegal. 2 by 3 times a length-3 vector is legal and produces length 2. Libraries that disagree with your hand check usually disagree on layout: rows vs columns, or whether the vector is on the left or the right. Fix layout. Do not “reshape until it runs” without checking one numeric example.

Another wrong picture is: “a linear layer is mysterious.” A linear layer in a neural net is y = W x + b (matrix-vector plus a bias vector). That is all. Each row of W is “how this output slot weights the inputs.” Each column is “how this input feature fans out.” Reading W is reading a bundle of dot products.

A third wrong picture is skipping shape logs. Garbage predictions with no exception often mean you swapped rows and columns and still multiplied something. Logging len(W) and len(W[0]) catches that.

The formula in words

Matrix-vector: for each row, dot that row with x. Stack those dots into a new list.

Bias: add a vector b of length rows, slot by slot, after the multiply.

Tiny numeric. Let

W = [[1, 0, 2], [0, 1, -1]], x = [1, 4, 3], b = [0.5, -0.5].

First row dot x: 11 + 04 + 23 = 7. Second: 01 + 14 + (-1)3 = 1. So W x = [7, 1]. Plus bias: [7.5, 0.5].

If x had length 2, the dots would be a shape error. Raise. Do not zip-and-truncate.

Why this is everywhere

A batch of embeddings stacked as rows is a matrix. Attention scores are tables of dot products (later: one line on attention). Projecting an embedding to a smaller space is a matrix with fewer rows than the embedding length. A linear classifier on frozen embeddings is W x + b on one vector. You can prototype that with lists, then swap in a real module.

Matrix-matrix multiply is the same idea with extra loops: each column of the second matrix is a vector you multiply by W. Skip it until you can do mat-vec without looking it up.

The transpose flips rows and columns. A 2 by 3 transpose is 3 by 2. If you stored W the wrong way, transpose is the fix — or, better, agree on layout in code review. A vector is a matrix with one column (or one row). The distinction is layout. Agree on layout.

Implement mat-vec

This 2 by 3 matrix maps a 3-d feature vector to a 2-d hidden vector. Print shapes as you go. That habit transfers when a real library complains about [2, 3] vs [3].

Run to execute this in your browser. Nothing is sent to a server.

You should see shape (2, 3), W x as [7.0, 1.0], and W x + b as [7.5, 0.5]. Hand-check: first component is 11 + 04 + 23 = 7, plus bias 0.5 → 7.5. Second is 01 + 14 + (-1)3 = 1, plus -0.5 → 0.5. If your library disagrees, your layout (rows vs columns) is wrong — not reality.

Note that dot here does not check lengths. mat_vec does, using the row length. That is the right place: one check per multiply.

Change x to length 2 and you should get ValueError. That is a good failure. A silent wrong length is a bad failure.

How agents use this

When a paper says the model projects queries, keys, and values, it means three matrices applied to the same token vectors. (Transformer internals stop at that one sentence.) When a framework “adds a linear classifier on frozen embeddings,” it is W x + b on the embedding of the last state. You can prototype that with lists.

• Tokens / features: a bag of flags plus a cosine plus a latency can be one vector x. A small W maps that to “search vs sql” logits. Shape of W is (number of tools, number of features).
• Ranking: a learned projection W so that cosine in the projected space matches your labels better. Apply W to every vector you compare, or to none.
• Loss: if y = W x + b and loss is a function of y, training will slope each entry of W (gradients lesson). The object you slope is this table of numbers.
• Logging: print len(W) and len(W[0]) in tests. Swapped rows and columns look like garbage predictions, not like an exception.

Stay in lists of floats. You do not need a GPU to see the contract: linear mix of inputs.

Note:A vector is a matrix with one column (or one row). The distinction is layout. Agree on layout.