Bias-Variance Tradeoff

This is one of the most important conceptual lectures in the course. We decompose prediction error into bias, variance, and irreducible noise, and show why training error alone is misleading. The U-shaped test error curve explains why more complex models are not always better. Learning curves help diagnose whether your problem needs more data, more features, or a simpler model. Cross-validation is positioned as the operational tool for managing this tradeoff.

Interactive: Bias-variance tradeoff

Drag the slider to change model complexity. Training error always decreases, but test error follows a U-shape — the bias-variance tradeoff.