← Lesson
Limitations
Joeven
Run
Reset
Python loads on first run
def naive_attn_cost(n): return n * n def retrieval_cost(chunks, chunk_len, query_len): return query_len + chunks * chunk_len print("tokens | attn n^2 | 20 chunks x 200 toks retrieve-ish") for n in (1024, 4096, 32768): print(n, naive_attn_cost(n), retrieval_cost(20, 200, n)) print("n^2 grows faster than selected chunks") print("even at 32768, 20*200 is a small add; the square is the monster")
Run to execute this in your browser. Nothing is sent to a server.