Intelligent Systems for Pattern Recognition - 9 CFU

Assignment Rules and Execution

The second midterm covers the probabilisti models introduced up to Lecture 14. To pass the midterm you should

1. perform one (only one) of the assignments described in the following;
2. report your results as a Colab notebook or a 10 slide presentation (both versions are equally fine) and upload it here by the (strict) deadline;

In case you are delivering a Colab notebook, please upload a txt file with the link to your Colab notebook (already ran). Note that Assignement 1 and Assigment 5 below have specialized delivery instructions.

You can use library functions to perform the analysis unless explicitly indicated. You can use whatever programming language you like, but I strongly suggest to use either Python or Matlab for which you have coding examples. Python will further allow you to deliver your midterm as a Colab notebook, but this is not a requirement (you can deliver a presentation instead). 

Your report (irrespectively of whether it is a notebook or a presentation) needs to cover at least the following aspects (different assignments might have different additional requirements):

1. A title with the assignment number and your name
2. The full code to run your analysis (for Colabs) or a few slides (for presentations) with code snippets highlighting the key aspects of your code
3. A section reporting results of the analysis and your brief comments on it
4. A final section with your personal considerations (fun things, weak aspects, possible ways to enhance the analysis, etc.).

Do not waste time and space to describe the dataset or the assignment you are solving as we will all be already informed on it.

N.B.: In order to deliver a Colab you might just upload a txt file containing the link to the Colab.

List of Midterm Assignments

Assignments are based on the following two datasets: 

DSET1 (Image processing): attached below or direct link here

DSET2 (Image processing: MNIST): (https://www.kaggle.com/datasets/hojjatk/mnist-dataset) or (https://huggingface.co/datasets/ylecun/mnist/) or (http://yann.lecun.com/exdb/mnist/)

Assignment 1

Derive the Forward-Backward algorithm for an HMM of order 2. Detail the step-by-step derivation of the algorithm starting from the initial formulation of the problem, motivating and justifying each mathematical manipulation (e.g. by referring to the appropriate rule or discussing the conditional independence assumption you are using).

The provide an implementation of the derived Forward-Backward algorithm. You do not need to develop the full code for a working HMM! Only write the piece of code which implements the Forward-Backward algorithm. Also you are NOT REQUIRED to run the code, only make sure to comment it by adding references to which equations from your derivation you are implementing with the specific relevan lines of your code.

To deliver this midterm you can opt to upload either:

• a single PDF report containing the derivations and the code
• a presentation containing the derivations and the code
• a scan of your handwritten derivations (provided that is legible) plus a pdf with the code
• anything that makes sense and is consistent with three examples above.

Assignment 2

Implement a simple image understanding application for DSET1 using the LDA model and the bag of visual terms approach described in Lecture 12-13. For details on how to implement the approach see the BOW demo and paper [12] referenced on the Moodle site. Keep one picture for each image subset (identified by the initial digit in the filename) out of training for testing. In short:

1.     Pick up one interest detector of your choice (it is ok to use those implemented in a library)

1.     For each image (train and test) extract the SIFT descriptors for the interest points identified by the your detector of choice

2.     Learn a K-dimensional codebook (i.e. run k-means with K clusters, with K chosen by you) from the SIFT descriptors of the training images (you can choose a subsample of them if k-means takes too long).

3.     Generate the bag of visual terms for each image (train and test): use the bag of terms for the training images to train an LDA model (use one of the available implementations). Choose the number of topics as you wish (but reasonably).

4.     Test the trained LDA model on test images: plot (a selection of) them with overlaid visual patches coloured with different colours depending on the most likely topic predicted by LDA.

Assignment 3

Implement from scratch an RBM and apply it to DSET2. The RBM should be implemented fully by you (both training and inference steps) but you are free to use library functions for the rest (e.g. image loading and management, etc.). Implement a generalization of the Contrastive Divergence (CD) learning algorithm that defines the number of steps K of the Gibbs sampling Markov chain runned before collecting the samples to estimate the model expectation. For instance the standard CD learning would be obtained with K=1. Test your models by training two versions of them, one with a small K and one with a medium K (I suggest you do not go over 10 steps), and discuss the differences in performance/behaviour (if any).

Outline of the assigment:

1.     Train an RBM with a number of hidden neurons selected by you (single layer) on the MNIST data (use the training set split provided by the website) using CD(K) with two choices of K.

2.     Use the trained RBMs to encode a selection of test images (e.g. using one per digit type) using the corresponding activation of the hidden neurons.

3.    Train a simple classifier (e.g. any simple classifier in scikit) to recognize the MNIST digits using as inputs their encoding obtained at step 2. Use the standard training/test split. Show a performance metric of your choice in the presentation/handout and use it to confront the two versions of the RBM (obtained with different K).

Once you have modelled the BN, also plug in the necessary local conditional probability tables. You can set the values of the probabilities following your own intuition on the problem (ie no need to learn them from data). Then run some episoded of Ancestral Sampling on the BN and discuss the results.

The assignment needs to be fully implemented by you, without using BN libraries.

Assignment 5

In this assignment, you will focus on recovering a causal graph from observational data and estimating the causal effect of a treatment variable on an outcome variable over a synthetic dataset. The assignement is detailed in the notebook provided below, which you will use as well to complete your assignment by uploading it here on Moodle.