Week: Lectures Recordings: Optimization | Computational Mathematics for Learning and Data Analysis - AA 2024/25 | INF - e-learning

Section outline

- Select activity Lecture 1.1 - introduction to the course
  
  Lecture 1.1 - introduction to the course File
- Select activity Lecture 1.2 - motivation for the course: four examples
  
  Lecture 1.2 - motivation for the course: four examples File
- Select activity Lecture 2.1: general notions of optimization
  
  Lecture 2.1: general notions of optimization File
- Select activity Lecture 2.2: starting very very easy and very slowly ramping up
  
  Lecture 2.2: starting very very easy and very slowly ramping up File
- Select activity Lecture 3.1: multivariate optimization: initial concepts, easy functions
  
  Lecture 3.1: multivariate optimization: initial concepts, easy functions File
- Select activity Lecture 3.2: "real" quadratic functions and how they work
  
  Lecture 3.2: "real" quadratic functions and how they work File
- Select activity Lecture 4.1: quadratic optimization: from optimality conditions to the gradient method
  
  Lecture 4.1: quadratic optimization: from optimality conditions to the gradient method File
- Select activity Lecture 4.2: the gradient method for quadratic functions, practice
  
  Lecture 4.2: the gradient method for quadratic functions, practice File
- Select activity Lecture 5.1: convergence rates: from the gradient method to the world
  
  Lecture 5.1: convergence rates: from the gradient method to the world File
- Select activity Lecture 5.2: sublinear convergence and where this leads us
  
  Lecture 5.2: sublinear convergence and where this leads us File
- Select activity Lecture 6.1: optimizing more general functions, but univariate ones
  
  Lecture 6.1: optimizing more general functions, but univariate ones File
- Select activity Lecture 6.2: first steps with local optimization: the role of derivatives
  
  Lecture 6.2: first steps with local optimization: the role of derivatives File
- Select activity Lecture 7.1: dichotomic search, from naive to model-based
  
  Lecture 7.1: dichotomic search, from naive to model-based File
- Select activity Lecture 7.2: faster local optimization and the role of models
  
  Lecture 7.2: faster local optimization and the role of models File
- Select activity Lecture 8.1: closing thoughts of univariate optimization, a fleeting glimpse to the global case
  
  Lecture 8.1: closing thoughts of univariate optimization, a fleeting glimpse to the global case File
- Select activity Lecture 8.2: theory of gradients and Hessians towards optimality conditions
  
  Lecture 8.2: theory of gradients and Hessians towards optimality conditions File
- Select activity Lecture 9.1: local first- and second-order optimality conditions (necessary and sufficient), convexity in \R^n
  
  Lecture 9.1: local first- and second-order optimality conditions (necessary and sufficient), convexity in \R^n File
- Select activity Lecture 10.1: the gradient method with "exact" line search
  
  Lecture 10.1: the gradient method with "exact" line search File
- Select activity Lecture 10.2: inexact line search, the Armijo-Wolfe conditions
  
  Lecture 10.2: inexact line search, the Armijo-Wolfe conditions File
- Select activity Lecture 11.1: convergence with the A-W LS, theory
  
  Lecture 11.1: convergence with the A-W LS, theory File
- Select activity Lecture 11.2: the A-W LS in practice
  
  Lecture 11.2: the A-W LS in practice File
- Select activity Lecture 12.1: "extremely inexact LS": fixed stepsize
  
  Lecture 12.1: "extremely inexact LS": fixed stepsize File
- Select activity Lecture 12.2: gradient twisting approaches at their best: Newton's method
  
  Lecture 12.2: gradient twisting approaches at their best: Newton's method File
- Select activity Lecture 13.1: all around Newton's method
  
  Lecture 13.1: all around Newton's method File
- Select activity Lecture 13.2: towards the very-large-scale, quasi-Newton methods
  
  Lecture 13.2: towards the very-large-scale, quasi-Newton methods File
- Select activity Lecture 14.1: deflected gradient methods I - Conjugate Gradient
  
  Lecture 14.1: deflected gradient methods I - Conjugate Gradient File
- Select activity Lecture 14.2: deflected gradient methods II - Heavy Ball
  
  Lecture 14.2: deflected gradient methods II - Heavy Ball File
- Select activity Lecture 15.1: the scary world of nondifferentiable optimization
  
  Lecture 15.1: the scary world of nondifferentiable optimization File
- Select activity Lecture 15.2: (convex) nondifferentiable optimization, converging against all odds
  
  Lecture 15.2: (convex) nondifferentiable optimization, converging against all odds File
- Select activity Lecture 16.1: better nondifferentiable approachess, as far as they can go
  
  Lecture 16.1: better nondifferentiable approachess, as far as they can go File
- Select activity Lecture 16.2: first steps on constrained optimization
  
  Lecture 16.2: first steps on constrained optimization File
- Select activity Lecture 17.1: algebraic representation of feasible sets, i.e., constraints
  
  Lecture 17.1: algebraic representation of feasible sets, i.e., constraints File
- Select activity Lecture 17.2: from the KKT conditions to duality
  
  Lecture 17.2: from the KKT conditions to duality File
- Select activity Lecture 18.1: first step in constrained optimization
  
  Lecture 18.1: first step in constrained optimization File
- Select activity Lecture 18.2: more (projected gradient) steps in constrained optimization
  
  Lecture 18.2: more (projected gradient) steps in constrained optimization File
- Select activity Lecture 19.1: from Frank-Wolfe to the dual method
  
  Lecture 19.1: from Frank-Wolfe to the dual method File
- Select activity Lecture 19.2: ending with a bang: the (primal-dual) interior-point method
  
  Lecture 19.2: ending with a bang: the (primal-dual) interior-point method File