One gradient step on a quadratic
Can a gradient step make the loss worse?
Interactive example
- Loss now
- 10
- Change in loss
- -9
A step in the negative-gradient direction can still increase loss when the step is too large.
See the values
| Quantity | Value |
|---|---|
| Current w | 0 |
| Gradient | -6 |
| Next w | 3 |
| Next loss | 1 |
What to notice
Relate a derivative, an update rule and the observed loss change.
Conditions
One-dimensional differentiable quadratic and the explicit update w − step × gradient.
What this example cannot show
A quadratic has a simple global shape; this does not describe all optimisation landscapes.
Sources and context
These sources provide topic context. The values and interaction are authored by AXIOVUM; no institutional endorsement is implied.