Linear Models for Prediction

Linear regression is your first credible baseline — transparent, fast, and fully inspectable. This lecture covers OLS regression, then introduces ridge and lasso as the first upgrade when you worry about overfitting or too many features. We discuss when to penalise, how to tune the regularisation parameter with cross-validation, and what the regularisation path tells you about your features.

Interactive: Regularisation path

Regularisation path: as λ increases (slider right), coefficients shrink toward zero. Stronger features resist longer. Ridge shrinks smoothly; lasso would send some coefficients exactly to zero.

Optional: technical supplement

For students who want the linear algebra under the hood: the matrix formulation of OLS, the normal equations and their geometry, why ridge mechanically repairs a singular X^⊤X — and how the same model, redrawn, is a one-neuron neural network.

Open the supplementary slides ↗ — optional and self-contained; nothing in it is required for the project.