Section outline
-
Schedule
Held in the first semester, September to December 2024. 72-hour, 9-CFU course.- Wed 16:00 - 18:00, Room Fib C1;
- Thu 11:00 - 13:00, Room Fib C;
- Fri 11:00 - 13:00, Room Fib M1.
Organisation
The course contains two modules:
- Optimization, taught by Antonio Frangioni: office hours Tuesday 9:00 - 11:00 also on this Team (code yh6612v)
- Numerical Linear Algebra, taught by Federico Poloni: office hours by appointment
However, the course is one and the same: one project, one exam, and both lecturers must always be contacted at the same time for any communication.
Online resources
- MS Team for online lectures and communication (access with code "0yqe9wv")
- Lectures log
Aims of the course
The course aims at providing the mathematical foundations for some of the main computational approaches to Learning, Data Analysis and Artificial Intelligence. These comprise techniques and methods for the numerical solution of systems of linear and nonlinear equations and related problems (e.g., computation of eigenvalues), as well as methods for the solution of constrained and unconstrained optimization problems. This requires the understanding of the connections between techniques of numerical analysis and optimization algorithms. The course focuses on presenting the main algorithmic approaches and the underlying mathematical concepts, with attention to the implementation aspects. Hence, use of typical mathematical environments (e.g., Matlab and Octave) and available solvers/libraries is discussed throughout the course.
Programme
- Linear algebra and calculus background
- Unconstrained optimization and systems of equations
- Direct and iterative methods for linear systems and least-squares
- Numerical methods for unconstrained optimization
- Iterative methods for computing eigenvalues
- Constrained optimization and systems of equations
- Duality (Lagrangian, linear, quadratic, conic, Fenchel's, ...)
- Numerical methods for constrained optimization
- Software tools for numerical computations (Matlab, Octave, ...)
- Sparse hints to AI/ML applications
Bibliography
- Slides and software by the lecturers available to students on this page
- Lecture notes for the optimization part are being prepared, the current partial form is distributed below
Useful books (referenced within the slides):
- L. N. Trefethen, D. Bau, Numerical Linear Algebra, SIAM, 1997
- J. Demmel, Applied Numerical Linear Algebra, SIAM, 1996
- L. Eldén, Matrix Methods in Data Mining and Pattern Recognition, 2007 (freely accessible from the Unipisa network)
- S. Boyd, L. Vandenberghe, Convex optimization, Cambridge University Press, 2008
- M.S. Bazaraa, H.D. Sherali, C.M. Shetty, Nonlinear programming: theory and algorithms, Wiley & Sons, 2006
- D.G. Luenberger, Y. Ye, Linear and Nonlinear Programming, Springer International Series in Operations Research & Management Science, 2008
- J. Nocedal, S. Wright, Numerical Optimization, Springer Series in Operations Research and Financial Engineering, 2006
- H.C. Pinkham. Analysis, Convexity, and Optimization, Draft of September 4, 2014 (available below, downloaded from here)
- Appunti di Ricerca Operativa (in Italian)
Other material (pointers to web resources) is suggested in the slides.Exam
The course requires a project, typically made in groups of two students, followed by an oral exam. Please check the "Projects" section for detailed information about the projects.Students are advised to submit the project incrementally to the lecturers, so that early problems can be spotted and weed out before a significant amount of work is wasted. There is no timeline for the projects: (partial) submissions can happen at any time. Once the project is completed and accepted, the date for the oral exam is freely chosen (with everyone's agreement). Please disregard any "appelli" you see in esami.unipi.it; they need be set (we have asked this to be avoided, but so far with no luck), but they are meaningless. All the students of a group are expected to take the oral exam in the same moment, although well-motivated exceptions are possible.