Lecture 5.1: convergence rates: from the gradient method to the world
Completion requirements
Our first convergence and efficiency proofs, what it reliably tells and what it does not (only a bound on the convergence speed, so the practice may be different). Plotting convergence rates in practice: the theory nails it quite well, but not perfectly (it's a bound, after all), in the positive definite case. Naming names: convergence rates (linear, sublinear, superlinear), what they mean, what they look like on the log-linear plot.