Train vs test
Fit on training data. Measure honest performance on held-out test data you never tuned on.
If you judge a model on the same data it was fitted to, you can look brilliant while failing in the wild. Split data:
- Train: learn parameters.
- Validation (optional but wise): choose hyperparameters / models.
- Test: final unbiased estimate of generalization.
Peeking at the test set
You try 50 architectures and always keep the one with best test accuracy. That “test” number is now optimistic — you peeked. Use validation for selection; touch the test set once.
Exam tip
Data leakage (test info sneaking into training) invalidates results.
Key takeaways
- Train = learn.
- Test = honest score.
- Never tune on the final test set.