Machine Learning quiz

24 questions from the Machine Learning track. Score 80% or higher to unlock a certificate.

What Is Machine Learning?
1. What is the core activity of machine learning?
- Writing a longer chain of if-statements
- Fitting a function from examples and measuring it on unseen data
- Buying a GPU
- Replacing tests with vibes
Features
2. What is a feature?
- The GPU brand
- A number (or list of numbers) you extract from an example for the model to see
- The test-set accuracy
- A Python comment
Data and Splits
3. When is a random shuffle a bad way to split agent data?
- Never; shuffle is always correct
- When examples are linked by time, user, or conversation, so the future or the same person leaks
- Only when the file is CSV
- When accuracy is already 100%
Baselines
4. Why report a majority-class baseline?
- It trains the neural net
- It shows the accuracy you get by always guessing the common label, so a real model has a bar to beat
- It removes leakage
- It replaces the test set
Supervised Learning
5. What is required for supervised learning?
- Unlabeled logs and a clustering algorithm
- An input and a target label for each training example
- A transformer
- Reinforcement rewards only
Loss Functions
6. Why train a classifier with cross-entropy instead of accuracy?
- Accuracy is illegal in Python
- Accuracy uses a hard decision and is not a smooth signal; cross-entropy penalizes low chance on the true class
- Cross-entropy always equals MSE
- Softmax deletes the true class
Gradient Descent
7. In the update w = w - lr * slope, why is there a minus sign?
- Python lists are 0-indexed
- Because we want to decrease the loss, so we move opposite the slope
- Gradients are always negative
- Minus makes the learning rate learn
A Linear Classifier
8. What does sigmoid do to a linear score?
- It picks a tool id
- It squashes the score into a chance between 0 and 1
- It splits the data into train and test
- It deletes negative features
Hyperparameters
9. Where should you pick the learning rate and the decision cutoff?
- On the test set, so the published number looks good
- On the validation set, then freeze them before you touch test
- On production traffic only
- Inside the loss formula
Overfitting
10. What is the signature of overfitting?
- Train and validation both terrible
- Train performance excellent, validation (or next week) much worse
- The learning rate is 0.01
- Softmax is used
Regularization
11. What is L2 regularization doing?
- Deleting the validation set
- Adding a cost for large weights so the model prefers a simpler function
- Making softmax faster
- Labeling unlabeled data
Metrics That Matter
12. A dataset is 95% class 0. A model always predicts 0. What is true?
- Precision and recall for class 1 are both excellent
- Accuracy is high, but recall for class 1 is zero
- MSE is undefined
- The confusion matrix has no TN
Rare Classes and Thresholds
13. You lower the yes-cutoff on a rare-class detector. What usually happens?
- Precision and recall both go to 1
- Recall tends to rise and precision tends to fall (more alarms, more catches)
- The training loss becomes MSE
- The test set gets bigger
Calibration
14. A model’s top bin has mean score 0.95 but only 50% true labels. What is that?
- Perfect calibration
- Overconfidence: the scores are too sure compared with reality
- Proof the loss was MSE
- A good reason to skip the test set
Ranking
15. The right chunk is 5th and the agent only sees top-3. What failed?
- Softmax overflow
- Recall at 3 (the gold item never entered the prompt)
- L2 regularization
- The learning rate
Embeddings
16. Why do we often use cosine similarity for embeddings?
- It is the only function Python can compute
- It compares direction (meaning) and ignores vector length
- It always equals accuracy
- It requires numpy
Nearest Neighbors
17. What does k-NN do at “training” time?
- It runs gradient descent for many epochs
- It stores the labeled points; the work happens at query time
- It deletes outliers
- It trains a softmax from scratch
Unsupervised Learning
18. What does k-means require you to choose up front?
- The correct label for every point
- The number of clusters k (and an initialization)
- A GPU
- Cross-entropy
A Tiny Neural Net
19. Why put a nonlinearity between two linear layers?
- To make Python faster
- Otherwise the composition is still one linear map and cannot learn many patterns
- ReLU deletes the gradient forever
- Softmax requires three layers
Train vs Inference
20. What are you doing when you call a hosted LLM with a prompt?
- Updating the model’s weights on your traces
- Inference: a frozen model runs a forward pass on your input
- Unsupervised k-means
- Gradient descent on the vendor’s GPU, billed to you as tokens
Drift
21. A router is 95% on the January test set and 60% on this week’s tickets. What should you do first?
- Delete the test set so the number looks better
- Label a fresh slice, compare it to January, and look for new words, tools, or policies
- Raise the learning rate
- Switch the loss to MSE
Rewards and Policies
22. What does a policy do?
- It stores the test set
- It maps a state (what you see) to an action, or to chances over actions
- It computes MSE
- It splits data by user
Rules, Prompts, or Train
23. When should you train a small router instead of writing a rule?
- Always; rules are obsolete
- When a simple rule cannot cover a large, stable, labeled remainder and a small model beats that rule on a frozen eval
- When you want to skip validation
- When the LLM API is free
Traces as a Dataset
24. Why freeze the list of test trace ids?
- So training can use the test set
- So two experiments are scored on the same exam and you can actually compare them
- So leakage is required
- So k-means can pick k