Supervised Learning

Module aims

Write a loss, a gradient step, and a validation protocol as three different objects, and refuse to treat training error as expected risk.

Course outline

1. 01 Objectives and models 20h

   Learning result Distinguish a prediction task, a model class, and a training objective.

   Formative check Four-option check

2. 02 Linear regression as an optimisation problem 20h

   Learning result Derive the normal equations from squared loss and say when the design is linear in parameters.

   Formative check Written derivation

3. 03 Logistic classification and likelihood 20h

   Learning result Derive binary cross-entropy from a Bernoulli likelihood.

   Formative check Written derivation

4. 04 Gradients and local descent 20h

   Learning result Justify the negative gradient as a local descent direction and state the first-order remainder.

   Formative check Written derivation

5. 05 Gradient descent and learning rates 15h

   Learning result For a one-dimensional quadratic, derive a constant-step interval that contracts.

   Formative check Written derivation

6. 06 Regularisation and inductive bias 15h

   Learning result Prove uniqueness of the ridge solution for a strictly positive penalty parameter even when X is rank deficient.

   Formative check Written derivation

7. 07 Validation, leakage and generalisation 20h

   Learning result Design a split that respects how new data arrive and exhibit a leakage failure.

   Formative check Counterexample

8. 08 Examination synthesis 20h

   Learning result Assemble objective, optimiser caveat, and evaluation into one short written argument.

   Formative check Written derivation

Learning results

Given a prediction task, specify a model class, a loss, an optimiser limitation, and a leakage-aware evaluation.

Subject skills

Empirical risk, linear and logistic models, gradients, ridge, train/validation/test splits.

Transferable skills

Report what a number measures; keep evaluation independent of fitting choices.

Related introductory topics

01Objectives and models3 topics

1. Empirical risk and loss functions
2. Linear regression as an optimisation problem
3. Logistic classification and likelihood

Chapter link

02Optimisation and regularisation3 topics

1. Gradients and local descent
2. Gradient descent and learning rates
3. Regularisation and inductive bias

Chapter link

03Evaluation and examination synthesis2 topics

1. Validation, leakage and generalisation
2. Supervised-learning examination synthesis

Chapter link