Quizzes

RAG & Memory quiz

24 questions from the RAG & Memory track. Score 80% or higher to unlock a certificate.

1. Why RAG

   1. Why do agents use RAG?

   • To avoid writing Python
   • To ground answers in a corpus you can update, authorize, and cite
   • Because cosine is legally required
   • To make prompts longer for fun

2. RAG vs a Lookup Tool

   2. When should you skip RAG?

   • Never; always retrieve
   • When a tool can look up a stable id or a database row
   • When the wiki is long
   • When cosine is 0.99

3. Ingest and Freshness

   3. What should you do with a draft page that contains “ignore previous instructions”?

   • Embed it so retrieval is complete
   • Skip it at ingest; do not chunk or embed it
   • Put it in the system prompt
   • Fine-tune on it

4. Chunking

   4. Why split on headings instead of only a character window?

   • Headings are required by JSON
   • Headings keep facts in their section so retrieval and citations make sense
   • Windows are illegal
   • To hide the original document

5. Overlap, Offsets, and Chunk Ids

   5. What must a citation be able to point at?

   • A random vector
   • A stable chunk id and, ideally, a byte range in the source
   • The system prompt
   • Yesterday’s Slack vibe

6. Embeddings and Retrieval

   6. What does cosine similarity measure for embeddings?

   • Disk speed
   • How aligned two vectors are (angle), used to rank chunks near a query
   • Whether JSON is valid
   • The number of GPUs

7. Scores and Thresholds

   7. What should you do when the best cosine is 0.12?

   • Stuff that chunk anyway
   • Treat it as a miss: no evidence, do not answer from junk
   • Fine-tune the model on pizza
   • Raise k to 50

8. Vector Indexes

   8. What does an ANN vector index buy you?

   • Perfect nearest neighbors for free
   • Lower latency by scoring a subset of vectors, at the cost of some recall
   • Automatic citations
   • Freedom from evals

9. Metadata Filters and Tenants

   9. How should a multi-tenant agent retrieve?

   • Search everyone, then hope the prompt is polite
   • Filter to the user’s tenant first, then rank
   • Embed tenant names only
   • Put all tenants in one chunk

10. Hybrid Search

    10. When is hybrid search worth it?

    • Never; vectors made keywords illegal
    • When queries mix paraphrase with exact tokens (ids, error codes, names)
    • Only for images
    • Only if you skip evals

11. Reciprocal Rank Fusion

    11. Why do people use RRF instead of adding raw BM25 to cosine?

    • RRF is a neural net
    • Ranks do not need the same numeric scale; 1/(k+rank) fuses lists safely
    • Cosine becomes exact
    • It cites automatically

12. Reranking

    12. What is reranking for?

    • Replacing retrieval entirely
    • Scoring a shortlist of chunks with a slower pair model so the prompt gets the precise evidence
    • Making embeddings longer
    • Skipping citations

13. MMR and Coverage

    13. What problem does MMR solve in RAG?

    • Faster GPUs
    • Near-duplicate chunks crowding out other evidence
    • Missing JSON schemas
    • HTTPS

14. Packing the Context Window

    14. How should you add retrieved chunks to a prompt?

    • Paste everything you fetched
    • Cap size, keep higher scores, label the block as data, mark truncation
    • Hide them in the system prompt as policy
    • Shuffle them randomly

15. Citations and Faithfulness

    15. What makes a citation real?

    • A number in brackets the model invented
    • A retrieved source plus a quote that actually appears in that source, checked by code
    • The model saying trust me
    • A longer answer

16. Refuse When Nothing Matches

    16. What should an agent do when retrieval scores are all below threshold?

    • Guess from training data and skip citations
    • Refuse: say nothing in the corpus matched
    • Retrieve 1,000 more chunks
    • Lower the threshold to zero forever

17. Retrieved Text Is Data

    17. Where should retrieved wiki text live in the prompt?

    • Inside the system prompt as new policy
    • In a labeled data block, treated as untrusted
    • As a new tool name
    • In the CSS

18. Agentic RAG

    18. What makes RAG agentic?

    • Using a purple vector database
    • The model (or policy) may retrieve, reformulate, retrieve again, or stop — retrieval is a tool in the loop
    • Skipping citations
    • Always retrieving 100 chunks

19. Query Rewrite

    19. What is query rewrite for?

    • Replacing the corpus
    • Turning sloppy user chat into terms the index can match
    • Skipping hybrid search
    • Making citations optional

20. Memory Types

    20. Which store should a versioned runbook live in?

    • Working memory (the live window only)
    • Episodic memory (Tuesday’s chat)
    • Semantic memory (curated facts), with the procedure compiled into tools
    • A single unsorted embedding of all Slack

21. Working Memory

    21. What belongs in working memory?

    • Every “thanks” embedded as a vector
    • A small structured state: goal, budget, last observation
    • The entire company wiki
    • Yesterday’s unrelated tickets

22. Memory Write-Back

    22. When may an agent write a new long-term fact from retrieved text?

    • Always; that is learning
    • Never from untrusted chunks; only from a trusted source or a human confirm
    • When cosine is high
    • When the page said please

23. Recall@k and RAG Evals

    23. What is the cheapest RAG eval?

    • Hire a larger model
    • A gold chunk id per question and recall@k (plus refuse on no-hit)
    • Production traffic only
    • Reading the README

24. When RAG Fails

    24. If recall@k is zero, what should you fix first?

    • The system prompt’s personality
    • Chunking, ingest, filters, or the retriever — the gold chunk never entered the prompt
    • A larger generator
    • Memory write-back