Overfitting

A model that memorizes the training set can fail on new data. That gap is overfitting.

Very flexible models can fit noise. Classic symptom: training loss near zero, test loss high.

As complexity grows, train error keeps falling while test error eventually rises.

Polynomial wiggles

Fit a degree-20 polynomial through 10 noisy points: it passes through every training point but oscillates wildly between them. A simpler line may have higher train error but much better test error.

Remedies: more data, simpler models, regularization, early stopping, dropout (in nets), cross-validation to detect the gap.

Key takeaways

Low train error ≠ a good model.
Watch the train–test gap.
Prefer models that work on unseen data.