We formalise the reinforcement learning problem by rooting it into Markov decision processes and we provide an overview of the main approaches to design reinforcement learning agents, including model-based, model-free, value and policy learning. We link classical approaches with modern deep learning based approximators (deep reinforcement learning). Methodologies covered include: dynamic programming, MC learning, TD learning, SARSA, Q-learning, deep Q-learning, policy gradient and deep policy gradient.
|
Date |
Topic |
References (OLD)
|
References (NEW)
| Additional Material
|
24
|
16/04/2024
(11-13)
|
Explicit Density Learning
explicit distribution models; neural ELBO; variational autoencoders
|
[DL] Sections 20.9, 20.10.1-20.10.3
|
[CHB] Section 19.2
[SD] Chapter 14 (generative learning), Chapter 17 (VAE)
| Additional Readings
[38] PixelCNN - Explict likelihood model
[39] Tutorial on VAE
Sofware
|
25
|
17/04/2024
(16-18)
|
Implicit models - Adversarial Learning
generative adversarial networks; wasserstein GANs; conditional generation; notable GANs; adversarial autoencoders
|
[DL] Section 20.10.4
|
[CHB] Chapter 17
[SD] Chapter 15 | Additional Readings
[40] Tutorial on GAN (here another online resource with GAN tips)
[40] Wasserstein GAN
[42] Tutorial on sampling neural networks
[43] Progressive GAN
[44] Cycle Gan
[45] Seminal paper on Adversarial AEs
Sofware
|
26
|
18/04/2024
(14-16)
|
Diffusion models
noising-denoising processes; kernelized diffusion; latent space diffusion; conditional diffusion models
|
Not covered
|
[CHB] Chapter 20
[SD] Chapter 18 | Additional Readings
[46] Introductory and survey paper on diffusion models
[47] Seminal paper introducing diffusion models
[48] An intepretation of diffusion models as score matching
[49] Paper introducing the diffusion model reparameterization
[50] Diffusion beats GAN paper
|
|
23-25/04/2024
|
NO LECTURE DURING THIS WEEK
|
|
|
|
27
|
30/04/2024
(11-13)
|
Normalizing flow models
probabilistic change of variable; forward/normalization pass; from 1D to multidimensional flows; survey of notable flow models; wrap-up of deep generative learning
|
Not covered
|
[CHB] Chapter 18
[SD] Chapter 16
| Additional Readings
[51] Survey paper on normalizing flows
[52] RealNVP paper
[53] GLOW paper
[54] MADE autoregressive flow
Sofware
|