1 
21/02/2019 (1416)

C1 
Introduction to the course: motivations and aim; course housekeeping (exams, timetable, materials); introduction to modern pattern recognition applications (slides)



2
 22/02/2019 (1113)
 L1
 Signal processing: timeseries; time domain analysis (statistics, correlation); spectral analysis; fourier analysis. (slides)

 Demo

3
 28/02/2019 (1416)
 C1
 Image processing: feature descriptors (color histograms, SIFT), spectral analysis, feature detectors (edge, blobs and segments). (slides)

 Additional Readings [1,2] Two highlevel surveys on visual feature extraction and representation

4
 01/03/2019 (1113)
 L1
 Generative and Graphical Models: probability refresher; graphical
model representation; directed and undirected models: Bayesian
networks; Markov networks; conditional independence and dseparation. (slides) [Video Bayesian Network Part2]
 [BRML] Ch. 1 and 2 (Refresher)
[BRML] Sect. 3.1, 3.2 and 3.3.1 (conditional independence) [BRML] Sect. 3.3 (Directed Models)
[BRML] Sect. 4.1, 4.2.04.2.2 (Undirected Models)
[BRML] Sect. 4.5 (Expressiveness)
 Sofware
 Pyro  Python library based on PyTorch
 PyMC3  Python library based on Theano
 Edward  Python library based on TensorFlow

5 
7/03/2019 (1416) 
C1 
Hidden Markov Models: learning in directed graphical models; forwardbackward algorithm; generative models for sequential data (slides) [Video HMM Part1]

[BRML] Sect. 23.1.0 (Markov Models)
[BRML] Sect. 23.2.023.2.4 (HMM and forward backward) 
Additional Readings [3] A classical tutorial introduction to HMMs 
6 
8/03/2019 (1113) 
L1 
Hidden Markov Models: part 2 Notes on the derivation of Jensen inequality for EM [Video HMM Part2] [Video HMM Part3]




14/03/2019 (1416) 

Lesson CANCELLED  Will be recovered in April




15/03/2019 (1113) 

Lesson CANCELLED  Will be recovered in April 


7 
21/03/2019 (1416) 
C1 
Markov Random Fields: learning in undirected graphical models; conditional random fields; pattern recognition applications (slides) [Video MRF Part1, Incomplete due to recording problems]

[BRML] Sect. 4.2.2, 4.2.5 (MRF) [BRML] Sect. 4.4 (Factor Graphs)
[BRML] Sect. 5.1.1 (Variable Elimination and Inference on Chain) [BRML] Sect. 9.6.0, 9.6.1, 9.6.4, 9.6.5 (Learning in MRF/CRF) 
Additional Readings [4,5] Two comprehensive tutorials on CRF ([4] more introductory and [5] more focused on vision) [6] A nice application of CRF to image segmentation
Sofware Check out pgmpy: it has Python notebooks to introduce to working with MRF/CRF 
8 
22/03/2019 (1113) 
L1 
Boltzmann Machines: bridging neural networks and generative models; stochastic neuron; restricted Boltzmann machine; contrastive divergence (slides) [Video MRF Part 2 + BM1]

[DL] Sections 20.1 and 20.2 
Additional Readings [7] A clean and clear introduction to RBM from its author
Sofware Matlab code for Deep Belief Networks (i.e. stacked RBM) and Deep Boltzmann Machines. 
9 
28/03/2019 (1416) 
C1 
Bayesian Learning: nonparametric models; variational learning (slides) [Video RBM] [Video Bayesian Learning]

[BRML] Sect. 11.2.1 (Variational EM), 20.420.6.1 (LDA) 
Additional Readings [8] LDA foundation paper [9] A gentle introduction to latent topic models [10] Foundations of bag of words image representation
Sofware

10 
29/03/2019 (1113) 
L1 
Bayesian Learning: sampling methods (Guest Lecture by Daniele Castellana) (slides)

[BRML] Sect. 27.127.3, 27.4.1, 27.6.2


M1 
01/04/2019 (13.3016) 
L1

Midterm 1 discussions



11 
08/04/2019 (1416) 
L1

Learning in Structured Domain I: Recursive Neural Networks (Guest lecture by Alessio Micheli) (slides)



12 
11/04/2019 (1416) 
C1 
Learning in Structured Domain II: Neural Networks for Graphs (Guest lecture by Alessio Micheli) (slides)



13 
12/04/2019 (1113) 
L1 
Randomized Recurrent Neural Networks (Guest lecture by Claudio Gallicchio) (slides)



M2 
29/04/2019 (13.3016)

O1

Midterm 2 discussions 


14 
02/05/2019 (1416) 
C1 
Convolutional Neural Networks (part I): introduction to CNN; basic CNN elements (slides)

[DL] Chapter 9 
Additional Readings [1115] Original papers for LeNet, AlexNet, VGGNet, GoogLeNet and ResNet. 
15 
03/05/2019 (1113) 
L1

Convolutional Neural Networks (part II): CNN architectures for
image recognition; convolution visualization; advanced topics
(deconvolution, dense nets); applications and code [Video CNN part 2]

[DL] Chapter 9 
Additional Readings [16] Complete summary of convolution arithmetics [17] Seminal paper on batch normalization [18] CNN interpretation using deconvolutions 
16 
06/05/2019 (1416) 
B 
Adversarial Attacks & Generative Adversarial Networks (Guest seminar by Jan Göpfert) (slides)



17 
09/05/2019 (1416) 
C1 
Deep Autoencoders: introduction to the deep learning module, sparse, denoising and contractive AE; deep RBM. (slides) [video]

[DL] Chapter 14, Sect 20.3, 20.4.0 (from 20.4.1 onwards not needed)

Additional Readings [19] DBN: the paper that started deep learning [20] Deep Boltzmann machines paper [21] Review paper on deep generative models [22] Long review paper on autoencoders from the perspective of representation learning [23] Paper discussing regularized autoencoder as approximations of likelihood gradient 
18 
10/05/2019 (1113) 
L1 
Gated Recurrent Networks: deep learning for sequence processing; gradient issues; longshort term memory; gated recurrent units; generative use of RNN (slides) [video]

[DL] Sections 10.110.3, 10.510.7, 10.10, 10.11 
Additional Readings [24] Paper describing gradient vanish/explosion [25] Original LSTM paper [26] An historical view on gated RNN [27] Gated recurren units paper [28] Seminal paper on dropout regularization
Sofware

19 
16/05/2019 (1416) 
C1 
Advanced Recurrent Architectures: sequencetosequence; attention models; multiscale network; memory networks; neural reasoning. (slides) [video]

[DL] Sections 10.12, 12.4.5 
Additional Readings [29,30] Models of sequencetosequence and imagetosequence transduction with attention [31,32] Models optimizing dynamic memory usage (clockwork RNN, zoneout) [33] Differentiable memory networks [34,35] Neural Turing Machines and followup paper on pondering networks [36] Transformer networks: a paper on the power of attention without recurrence

20 
17/05/2019 (1113) 
L1 
PyTorch – neural networks in Python: python; pytorch; RNN (Coding practice by Antonio Carta) (slides)



21 
23/05/2019 (1416) 
C1 
An introduction to Tensorflow and Keras
python; numpy, tensorflow, keras; (Coding practice by Luca Pedrelli) (slides)



22 
24/05/2019 (1113) 
L1 
Generative and Unsupervised Deep Learning: explicit distribution models; variational autoencoders; adversarial learning. (slides) [Video part1 VAE] [Video part2 GAN]

[DL] Sections 20.9, 20.10.120.10.4 
Additional Readings [37] PixelCNN  Explict likelihood model [38] Tutorial on VAE [39] Tutorial on GAN (here another online resource with GAN tips) [40] Wasserstein GAN [41] Tutorial on sampling neural networks
Sofware

23 
30/05/2019 (1416) 
C1 
Advanced research topics: new recurrent models, graph neural networks, tensorbased neural models and structured output prediction (Quick seminars by Antonio Carta, Daniele Castellana, Federico Errica and Marco Podda) [Slides deep learning for graphs]
 


31/05/2019 (1113) 
L1 
Lesson cancelled due to strike



24 
03/06/2019 (1618)

F1 
Final lecture: course wrapup; research themes; final projects; exam modalities 


M3 
06/06/2018 (13.3016.30)

L1 
Midterm 3 discussions


