Section outline
-
The module presents the fundamental concepts, challenges, architectures and methodologies of deep learning. We introduce the learning of neural representations from data of heterogenous nature (vectorial, image, sequential), discussing inductive biases for each data type along with relevant foundational architectures. We will discuss key concepts to understand and address issues in deep neural architectures, with focus on information propagation and stability of learning processes. Models covered in this module include: convolutional neural networks, long-short term memory, gated recurrent units, randomized networks, sequence-to-sequence, neural attention, transformers, neural ODEs. Methodological lectures will be complemented by introductory seminars to Keras-TF and Pytorch.
Date Topic References Additional Material 15 23/03/2026
(11-13)Convolutional Neural Networks I
Introduction to the deep learning module; introduction to CNN; basic CNN elements[SD] Chapter 10 Additional Readings
[6-10] Original papers for LeNet, AlexNet, VGGNet, GoogLeNet and ResNet.
16 24/03/2026
(16-18)Convolutional Neural Networks II
CNN training, notable CNN architectures; advanced topics (deconvolution, causal convolutions, dilated convolutions); vision tasks with CNNs[SD] Chapter 10 (CNNs)
SD] Chapter 11 (residual nets)
Additional Readings
[11] Complete summary of convolution arithmetics[12] Seminal paper on batch normalization
[13] Seminal paper on dilated convolutions
[14] Object detection by Faster RCNN
16b 25/03/2026
(11-13)Lecture time reduced by assessment student survey.
Remaining time dedicated to conclude CNN lecture.
17 31/03/2026
(11-13)Information Propagation in Deep Networks
sequential data processing; RNNs refresher; exploding and vanishing gradient problem; information propagation beyond the gradient
Additional Readings
[15] Paper describing gradient vanish/explosion18 01/04/2026
(16-18)Advanced Recurrent Models
gating neurons; LSTM, GRU; randomized RNNs; autoregressive modeling with RNNs
Coverage of Prince book on this lecture is inadequate (for reasons oblivious to me). You can use the course slides for this topic, and if you like you can integrate those with chapter 10 from the Deep Learning Book. Additional Readings
[16] Original LSTM paper[17] An historical view on gated RNN
[18] Gated recurrent units paper
[19] Seminal paper on dropout regularization
Software
- A simple introduction to generative use of LSTM
02/04/2026-07/04/2026 - EASTER BREAK
Lectures resume on: 08/04/2026
19 08/04/2026
(16-18)Attention-based architectures I: recurrent encoder-decoder
sequence-to-sequence task; encoder-decoder architectures; cross-attention mechanism; general view on neural attention
Coverage of Prince book on this lecture is inadequate. You can use the course slides. Additional Readings
[20,21] Seminal papers on encoder-decoder architectures with cross-attention
20 09/04/2026
(11-13)Attention-based architectures II: Transformers
self-attention; transformer models; inductive bias; gradient propagation; self-supervised training
[SD] Chapter 12 Additional Readings
[22] Seminal paper on Transformers
[23] Transformers in vision21 14/04/2026
(11-13)Coding practice I - Lecture by Riccardo Massidda
22 15/04/2026
(16-18)Coding practice II - Lecture by Riccardo Massidda