Intelligent Systems for Pattern Recognition - 9 CFU
The course is held on the second term. The tentative schedule for A.A. 2022/23 is provided in table below.
The first lecture of the course will be ON THE WEEK OF FEBRUARY 20-24 (exact date TBD). The course will be hybrid, both in person and online on the dedicated MS Team.
Recordings of the lectures will be made available to the students following the course.
Day Time Tuesday 11.00-12.45 (Room C1) Wednesday 16.15-18.00 (Room E) Thursday 14.15-16.00 (Room C)
Course prerequisites include knowledge of machine learning fundamentals (e.g. covered through ML course). Knowledge of elements of probability and statistics, calculus and optimization algorithms are also expected. Previous programming experience with Python is a plus for the practical lectures.
The course introduces students to the analysis and design of advanced machine learning and deep learning models for modern pattern recognition problems and discusses how to realize advanced applications exploiting computational intelligence techniques.
The course is articulated in five parts. The first part introduces basic concepts and algorithms concerning traditional pattern recognition, in particular as pertains sequence and image analysis. The next two parts introduce advanced models from two major learning paradigms, that are deep neural networks and generative models, their combinaton and their use in pattern recognition applications. The fourth part will introduce fundamentals of reinforcement learning and deep reinforcement learning. The final part of the course will present selected recent works, models and applications of learning models.
Presentation of the theoretical models and associated algorithms will be complemented by introductory classes on the most popular software libraries used to implement them.
The course hosts guest seminars by national and international researchers working on the field as well as by companies that are engaged in the development of advanced applications using machine learning models.
The official language of the course is English: all materials, references and books are in English. Lecture slides will be made available here, together with suggested readings.
Topics covered -Bayesian learning, graphical models, learning with sampling and variational approximations, fundamentals of deep learning (CNNs, AE, DBN, GRNs), deep learning for machine vision and signal processing, advanced deep learning models (transformers, NTMs), generative deep learning (VAE, GANs, diffusion models, score-based models) deep graph networks, reinforcement learning and deep reinforcement learning, signal processing and time-series analysis, image processing, filters and visual feature detectors, pattern recognition applications (machine vision, bio-informatics, robotics, medical imaging, etc), introduction to programming libraries and frameworks.
Textbooks and Teaching Materials
The course does not have an official textbook covering all its contents. However, a list of reference books covering parts of the course is listed at the bottom of this section (note that all of them have an electronic version freely available online).
[BRML] David Barber, Bayesian Reasoning and Machine Learning, Cambridge University Press (PDF)
[DL] Ian Goodfellow and Yoshua Bengio and Aaron Courville , Deep Learning, MIT Press (ONLINE)
[RL] Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, Second Edition, MIT Press, 2018 (PDF)
Introduction (2h)Introduction to the course philosophy, its learning goals and expected outcomes. We will discuss prospectively the overall structure of the course and the interelations between its parts. Exam modalities and schedule are also discussed.
Date Topic References Additional Material 1 Introduction to the course
Motivations and aim; course housekeeping (exams, timetable, materials); introduction to modern pattern recognition applications
Fundamentals of Pattern Recognition (6h)The module will provide a brief introduction to classical pattern recognition for signal/timeseries and for images. We will cover approaches working on the spatial (temporal) and frequency (spectral) domain, presenting methods to represent temporal and visual information in static descriptors, as well as approaches to identify relevant patterns in the data (feature descriptors). Methodologies covered include correlation analysis, Fourier analysis, wavelets, intensity gradient-based descriptors and detectors, normalized cut segmentation.
Date Topic References Additional Material 2 16/02/2022
Timeseries; time domain analysis (statistics, correlation); spectral analysis; fourier analysis.
Image Processing I
Spatial feature descriptors (color histograms, SIFT); spectral analysis.
 Survey on visual descriptors
Image Processing II
Feature detectors (edge, blobs); image segmentation; wavelet decompositions
 Survey on visual feature detectors
A wavelet browser to visualize some popular wavelet families and their instances, powered by the PyWavelet library.
Generative Learning (20h)
The module introduces probabilistic learning, causal models, generative modelling and Bayesian learning. We will discuss fundamental algoritms and concepts, including Expectation-Maximization, sampling and variational approximations, and we will study relevant models from the three fundamental paradigms of probabilistic learning, namely Bayesian networks, Markov networks and dynamic models. Models covered include: Bayesian Networks, Hidden Markov Models, Markov Random Fields, Boltzmann Machines, Latent topic models.
Date Topic References Additional Material 5 23/02/2022
Introduction to Generative and Graphical Models.
Probability refresher; graphical model representation; directed and undirected models
[BRML] Ch. 1 and 2 (Refresher)
[BRML] Sect. 3.1, 3.2 and 3.3.1
Conditional independence and causality - Part I
Bayesian networks; Markov networks; conditional independence;
[BRML] Sect. 3.3 (Directed Models)
[BRML] Sect. 4.1, 4.2.0-4.2.2 (Undirected Models)
[BRML] Sect. 4.5 (Expressiveness)
Conditional independence and causality - Part II
d-separation; structure learning in Bayesian Networks
[BRML] Sect. 9.5.1 (PC algorithm)
[BRML] Sect. 9.5.2 (Independence testing)
[BRML] Sect. 9.5.3 (Structure scoring)
 A short review of BN structure learning
 PC algorithm with consistent ordering for large scale data
 MMHC - Hybrid structure learning algorithm
- A selection of BN structure learning libraries in Python: pgmpy, bnlearn, pomegranate.
- bnlearn: the most consolidated and efficient library for BN structure learning (in R)
- Causal learner: a mixed R-Matlab package integrating over 26 BN structure learning algorithms.
Hidden Markov Models - Part I
learning in directed graphical models; forward-backward algorithm; generative models for sequential data
[BRML] Sect. 23.1.0 (Markov Models)
[BRML] Sect. 23.2.0-23.2.4 (HMM and forward backward)
 A classical tutorial introduction to HMMs
Hidden Markov Models - Part II
EM algorithm, learning as inference, Viterbi Algorithm
[BRML] Sect. 23.2.6 (Viterbi)
[BRML] Sect. 23.3.1-23.3.4 (EM and learning)
HMM III + Markov Random Fields
learning in undirected graphical models; conditional random fields; pattern recognition applications
[BRML] Sect. 4.2.2, 4.2.5 (MRF)
[BRML] Sect. 4.4 (Factor Graphs)
[BRML] Sect. 5.1.1 (Variable Elimination and Inference on Chain)
[BRML] Sect. 9.6.0, 9.6.1, 9.6.4, 9.6.5 (Learning in MRF/CRF)
[7,8] Two comprehensive tutorials on CRF ( more introductory and  more focused on vision)
 A nice application of CRF to image segmentation
Check out pgmpy: it has Python notebooks to introduce to working with MRF/CRF
Bayesian Learning I
Principles of Bayesian learning; EM algorithm objective; principles of variational approximation
[BRML] Sect. 11.2.1 (Variational EM) 12 10/03/2022
Bayesian Learning II
latent topic models; Latent Dirichlet Allocation; machine vision application of latent topic models
[BRML] Sect. 20.4-20.6.1 (LDA) Additional Readings
 LDA foundation paper
 A gentle introduction to latent topic models
 Foundations of bag of words image representation
Bayesian Learning III
sampling methods; ancestral sampling; Gibbs sampling and Monte Carlo methods
Guest lecture by Daniele Castellana
[BRML] Sect. 27.1-27.3, 27.4.1, 27.6.2 Additional Readings
 A step-by-step derivation of collapsed Gibbs sampling for LDA
bridging neural networks and generative models; stochastic neuron; restricted Boltzmann machine; contrastive divergence
[DL] Sections 20.1 and 20.2 Additional Readings
 A clean and clear introduction to RBM from its author
Matlab code for Deep Belief Networks (i.e. stacked RBM) and Deep Boltzmann Machines.
Deep Learning (24h)
The module presents the fundamental concepts, challenges, architectures and methodologies of deep learning. We introduce the learning of neural representations from vectorial, sequential and image data, covering both supervised and unsupervised learning, and hinting at various forms of weak supervision. We close the gap between neural networks and probabilistic learning by discussing generative deep learning models. Models covered include: deep autoencoders, convolutional neural networks, long-short term memory, gated recurrent units, deep reservoir computing, sequence-to-sequence, neural attention, neural Turing machines, variational autoencoders, generative adversarial networks. Methodological lectures will be complemented by introductory seminars to Keras-TF and Pytorch.
Date Topic References Additional Material 15 16/03/2022
Convolutional Neural Networks I
Introduction to the deep learning module; introduction to CNN; basic CNN elements
[DL] Chapter 9 Additional Readings
[15-19] Original papers for LeNet, AlexNet, VGGNet, GoogLeNet and ResNet.
Convolutional Neural Networks II
CNN architectures for image recognition; convolution visualization; advanced topics (deconvolution, dense nets); applications and code
[DL] Chapter 9 Additional Readings
 Complete summary of convolution arithmetics
 Seminal paper on batch normalization
 CNN interpretation using deconvolutions
 CNN interpretation with GradCAM
LECTURE CANCELLED (RECOVERED ON THE 14th MARCH) 23/03/2022
LECTURE CANCELLED (RECOVERED ON THE 25th MARCH) 24/03/2022
LECTURE CANCELLED (RECOVERED ON THE 28th MARCH) 17 25/03/2022
ROOM D1 + ONLINE
Sparse, denoising and contractive AE; deep RBM
[DL] Chapter 14, Sect 20.3, 20.4.0 (from 20.4.1 onwards not needed) Additional Readings
 DBN: the paper that started deep learning
 Deep Boltzmann machines paper
 Review paper on deep generative models
 Long review paper on autoencoders from the perspective of representation learning
 Paper discussing regularized autoencoder as approximations of likelihood gradient
Gated Recurrent Networks I
Deep learning for sequence processing; gradient issues;
[DL] Sections 10.1-10.3, 10.5-10.7, 10.10, 10.11 Additional Readings
 Paper describing gradient vanish/explosion
 Original LSTM paper
 An historical view on gated RNN
 Gated recurren units paper
 Seminal paper on dropout regularization
Gated Recurrent Networks II
long-short term memory; gated recurrent units; generative use of RNN
Sofware 20 30/03/2022
Coding practice I - Tensorflow 21 31/03/2022
Coding practice II - PyTorch 22 05/04/2022
Deep Randomized Networks - Guest lecture by Claudio Gallicchio
reservoir computing; randomized models; echo state networks
Advanced Recurrent Architectures and Attention
sequence-to-sequence; attention models; multiscale network; hierarchical models
[DL] Sections 10.12, 12.4.5 Additional Readings
[34,35] Models of sequence-to-sequence and image-to-sequence transduction with attention
[36,37] Models optimizing dynamic memory usage (clockwork RNN, zoneout)
 Transformer networks: a paper on the power of attention without recurrence
memory networks; neural Turing machines
 Differentiable memory networks
[40,41] Neural Turing Machines and follow-up paper on pondering networks
Unsupervised and Generative Deep Learning I
explicit distribution models; neural ELBO; variational autoencoders
[DL] Sections 20.9, 20.10.1-20.10.3 Additional Readings
 PixelCNN - Explict likelihood model
 Tutorial on VAE
Unsupervised and Generative Deep Learning II
generative adversarial networks; adversarial autoencoders
[DL] Section 20.10.4 Additional Readings
 Tutorial on GAN (here another online resource with GAN tips)
 Wasserstein GAN
 Tutorial on sampling neural networks
 Progressive GAN
 Cycle Gan
 Seminal paper on Adversarial AEs
Reinforcement Learning (14h)We formalise the reinforcement learning problem by rooting it into Markov decision processes and we provide an overview of the main approaches to design reinforcement learning agents, including model-based, model-free, value and policy learning. We link classical approaches with modern deep learning based approximators (deep reinforcement learning). We overview the main programming frameworks available. Methodologies covered include: dynamic programming, MC learning, TD learning, SARSA, Q-learning, deep Q-learning, policy gradient and deep policy gradient, MC tree search.
Date Topic References Additional Material 28 21/04/2022
Reinforcement learning fundamentals
reinforcement learning problems; environment; agent; actions and policies; taxonomy of approaches
[RL] Chapter 1 Software
Open AI gym for RL environments and tasks
Markov Decision Processes
formal model of RL probelms; rewards; returns; Bellman expectation and optimality
[RL] Chapter 3 30 27/04/2022
dynamic programming; policy evaluation; policy iteration; value iteration
[RL] Chapter 4 Software
Model-free reinforcement learning
model-free predition; model-free control; Monte Carlo methods; TD learning; SARSA; Q-learning
[RL] Section 5.1-5.6, 6.1-6.6, 7.1, 7.2, 12.1, 12.2, 12.7
 The original Q-learning paper
NO LECTURE 04/05/2022
NO LECTURE 32 05/05/2022
linear incremental methods; batch value function approximation; deep Q-learning; linear least-squares control
[RL] Section 9.1-9.5, 9.8, 10.1, 11.1-11.5 Additional Reading:
 Original DQN paper
 Double Q-learning
 Dueling Network Architectures
 Prioritized Replay
(16-18 - ROOM C)
Policy gradient methods [RL] Chapter 13 Additional Reading:
 Original REINFORCE paper
 Learning with the actor-critic architecture
 Accessible reference to natural policy gradient
 A3C paper
 Deep Deterministic Policy Gradient
 TRPO paper
 PPO paper
Integrating Learning and Planning [RL] Chapter 8, Sect 16.6 Additional Reading:
 UCT paper: the introduction of Monte-Carlo planning
 MoGo: the grandfather of AlphaGo (RL using offline and online experience)
 AlphaGo paper
 AlphaGo without human bootstrap
Advanced Topics and Applications (8h)
The module covers some recent and interesting development and research topics in the field of machine learning. Topics choice is likely to vary at each edition. Example topics include: deep learning for graphs, learning with structured data, continual learning, distributed learning, learning-reasoning integration, edgeAI,etc.. The module concludes with a final lecture which discusses the course content retrospectively and details the exam modalities, topics and deadlines.
Date Topic References Additional Material 27 20/04/2022
Continual Learning - Guest lecture by Vincenzo Lomonaco 35 11/05/2022
Deep learning for graphs Software
- PyDGN: our in-house DLG library
- PyTorch geometric
- Deep graph library
[66-67] Seminal works on neural networks for graphs
 Recent tutorial paper
Final lecture 37 20/05/2022
(11-13) ROOM D1
Research seminars by Ph.D. students
Andrea Cossu - The reasonable effectiveness of pre-trained models in Continual Learning
Michele Resta - Continual Incremental Language Learning for Neural Machine Translation
RIccardo Massidda - Ontology-Driven Semantic Alignment of Artificial Neurons
Danilo Numeroso - Neural Algorithmic Reasoning
Francesco Landolfi - Graph Pooling with Maximum Weight k-Independent Sets
Course Grading and Exams
Typical course examination (for students attending the lectures) is performed in 2 stages: midterm assignments and an oral exam. Midterms waive the final project.
Midterms consist in short assignments involving with one of the following tasks:
- A quick and dirty (but working) implementation of a simple pattern recognition algorithm
report concerning the experience of installing and running a demo
application realized using available deep learning and machine learning
- A summary of a recent research paper on topics/models related to the course content.
The midterms can consist in either the delivery of code (e.g. colab notebook) or a short slide deck (no more than 10 slides) presenting the key/most-interesting aspects of the assignment.
Students might be given some amount of freedom in the choice of assignments, pending a reasonable choice of the topic. The assignments will roughly be scheduled every 3/4 weeks.
The oral examination will test knowledge of the course contents (models, algorithms and applications).
Exam Grading (with Midterms)
The final exam vote is given by the oral grade. The midterms only wave the final project but do not contribute to the grade. In other words you can only fail or pass a midterm. You need to pass all midterms in order to succesfully wave the final project.
Alternative Exam Modality (No Midterms / Non attending students)
Working students, those not attending lectures, those who have failed midterms or simply do not wish to do them, can complete the course by delivering a final project and an oral exam. Final project topics will be released in the final weeks of the course: contact the instructor by mail to arrange choice of the topics once these are published.
The final project concerns preparing a report on a topic relevant to the course content or the realization of a software implementing a non-trivial learning model and/or a PR application relevant for the course. The content of the final project will be discussed in front of the instructor and anybody interested during the oral examination. Students are expected to prepare slides for a 15 minutes presentation which should summarize the ideas, models and results in the report. The exposition should demonstrate a solid understanding of the main ideas in the report.
Grade for this exam modality is determined as
whereis the project grade and is the oral grade
Midterms and Projects