Section outline

◄Fundamentals of Pattern Recognition (6h)
►Deep Learning (18h)

  • The module introduces probabilistic learning, causal models, generative modelling and Bayesian learning. We will discuss fundamental algoritms and concepts, including Expectation-Maximization, sampling and variational approximations, and we will study relevant models from the three fundamental paradigms of probabilistic learning, namely Bayesian networks, Markov networks and dynamic models.  Models covered include: Bayesian Networks, Hidden Markov Models, Markov Random Fields, Boltzmann Machines,  Latent topic models.


    		 Date Topic  References   (OLD)
    		  References (NEW)    Additional Material 
      5 
    		05/03/2024
    (11-13)
    		 Introduction to Generative Graphical Models.
    P    robability refresher; graphical model representation; directed and undirected models
    		 [BRML] Ch. 1 and 2 (Refresher)
    [BRML] Sect. 3.1, 3.2 and 3.3.1
    (conditional independence)
    		  [CHB] Sect. 2.1-2.4 (refresher)
     [CHB] Sect. 2.5 (ML probabilities)
     [CHB] Sect. 11.1 and Sect. 11.2.1 (graphical models + conditional independence)
    		 Software
    • Pyro - Python library based on PyTorch
    • PyMC3 - Python library based on Theano
    • Edward - Python library based on TensorFlow
    • TensorFlow Probability - Probabilistic models and deep learning in Tensorflow

    		06/03/2024
    (16-18)
    		 LECTURE CANCELLED DUE TO STUDENT ASSEMBLY
    		 
    6
    		 07/06/2024
    (14-16)
    		 Conditional independence and causality - Part I
    Bayesian networks; Markov networks; conditional independence;
    		 [BRML] Sect. 3.3 (Directed Models)
    [BRML] Sect. 4.1, 4.2.0-4.2.2 (Undirected Models)
    [BRML] Sect. 4.5 (Expressiveness)
    		  [CHB] 11.1-11.3, 11.6 Graphical Models
     [CHB] 11.2 Conditional Independence

    Disclaimer: Coverage of the Bishop book on this lecture is partial. I suggest to use Barber's Book.
    7
    		 08/03/2023
    (14-16)
    AULA L1
    RECOVERY LECTURE
    		 Conditional independence and causality - Part II
    d-separation; structure learning in Bayesian Networks
    		 [BRML] Sect. 9.5.1 (PC algorithm)
    [BRML] Sect. 9.5.2 (Independence testing)
    [BRML] Sect. 9.5.3 (Structure scoring)
    		  Disclaimer: Coverage of the Bishop book on this lecture is inadequate. I suggest to use Barber's Book.  Additional readings
    [3] A short review of BN structure learning
    [4] PC algorithm with consistent ordering for large scale data
    [5] MMHC - Hybrid structure learning algorithm

    If you are interested in deepening of your knowledge on causality this is an excellent book (also freely available online): Jonas Peters, Dominik Janzing, Bernhard Schölkopf, Elements of causal inference : foundations and learning algorithms, MIT Press.

    Software
    - A selection of BN structure learning libraries in Python: pgmpy, bnlearn, pomegranate.
    - bnlearn: the most consolidated and efficient library for BN structure learning (in R)
    - Causal learner: a mixed R-Matlab package integrating over 26 BN structure learning algorithms.
    8
    		 12/03/2024
    (11-13)
    		 Hidden Markov Models  - Part I
    learning in directed graphical models; forward-backward algorithm;  generative models for sequential data
    		  [BRML] Sect. 23.1.0 (Markov Models)
    [BRML] Sect. 23.2.0-23.2.4 (HMM and forward backward) 
    		 [CHB] 11.3 Sequence models

    Coverage of the Bishop book on this lecture is inadequate. I suggest to use Barber's Book.

    		13/03/2024
    (16-18)
    		 LECTURE CANCELLED (RECOVERY LECTURE ON FRIDAY)
    		 
    9
    		 14/03/2023
    (14-16)
    		 Hidden Markov Models - Part II
    EM algorithm, learning as inference, Viterbi Algorithm
    		 [BRML] Sect. 23.2.6 (Viterbi)
    [BRML] Sect. 23.3.1-23.3.4 (EM and learning)
    		  Coverage of the Bishop book on this lecture is inadequate. I suggest to use Barber's Book. Additional Readings
    [6]  A classical tutorial introduction to HMMs
    10
    		 15/03/2023
    (14-16)
    AULA L1
    RECOVERY LECTURE
    		 Markov Random Fields I
    learning in undirected graphical  models;
    		[BRML] Sect. 4.2.2, 4.2.5 (MRF)
    [BRML] Sect. 4.4 (Factor Graphs)

    		 Coverage of the Bishop book on this lecture is inadequate. I suggest to use Barber's Book.
    11
    		 19/03/2024
    (11-13)
    		 Markov Random Fields II
    conditional random fields; pattern recognition applications
    		 [BRML] Sect. 5.1.1 (Variable Elimination and Inference on Chain) 
    [BRML] Sect. 9.6.0, 9.6.1, 9.6.4, 9.6.5 (Learning in MRF/CRF)
    		 Coverage of the Bishop book on this lecture is inadequate. I suggest to use Barber's Book.
    		 Additional Readings
    [7,8] Two comprehensive tutorials on CRF ([7] more introductory and [8] more focused on vision)
    [9] A nice application of CRF to image segmentation

    Sofware
    12
    		 20/03/2024
    (16-18)
    		 Bayesian Learning I
    Principles of Bayesian learning; EM algorithm objective; principles of variational approximation; latent topic models; Latent Dirichlet Allocation (LDA).
    		 BRML] Sect. 11.2.1 (Variational EM)
    		 [CHB] 15.4 Evidence Lower Bound and the generalized EM
    13 21/03/2024
    (14-16)
    		 Bayesian Learning II
    LDA learning; machine vision application of latent topic models;

    Bayesian Learning III
    sampling methods; ancestral sampling;
    		 [BRML] Sect. 20.4-20.6.1  (LDA)

    [BRML] Sect. 27.1 (sampling), Sect. 27.2 (ancestral sampling)
    Bishop's book does not cover LDA: I suggest to use Barber's Book for this.

    [CHB] 14.1.1-2 (Sampling) 14.2.5 (ancestral)
    		   Additional Readings
    [10] LDA foundation paper
    [11] A gentle introduction to latent topic models
    [12] Foundations of bag of words image representation

    Sofware
    14
    		26/04/2024
    (11-13)    		 Bayesian Learning III
    Gibbs sampling

    Boltzmann Machines
    bridging neural networks and generative models; stochastic neuron; restricted Boltzmann machine; contrastive divergence and Gibbs sampling in use
    		 [BRML] Sect. 27.3 (Gibbs sampling)

    [DL] Sections 20.1 and 20.2 (RBM)
    		  [CHB] 14.2.4 (Gibbs)

    Bishop's book does not cover RBMs: the slides (possibly integrated by reference [14]) are enough for this part.
    		  Additional Readings
    [13] A step-by-step derivation of collapsed Gibbs sampling for LDA
    [14] A clean and clear introduction to RBM from its author

    Sofware
    Matlab code for Deep Belief Networks (i.e. stacked RBM) and Deep Boltzmann Machines.

◄Fundamentals of Pattern Recognition (6h)
►Deep Learning (18h)