HLT - 2023

Section outline

• Select section

  Collapse all Expand all

  Online lectures and their recordings are available on this Teams:
  
  

  649AA 22/23 - Human Language Technologies

  
  

  Those who do not have a University account, please write me and I will send you an invitation link.
  Please turn off your camera and microphone. Use the chat to ask questions.

  
  

  Schedule

  Day	Hour	Room
  Monday	14:15-16	M1
  Tuesday	9:15-11	M1
  Wednesday	14:15-16	M1

  Jupyter Notebook Server

  A server is available for running Jupyter Notebooks. You can log into the server using your University credentials (do not use your private Gmail account).

  Each user has his/her own home directory at:

       HLT/Home/<email@studenti.unipi.it>

  Jupyterlab is also integrated with GithHub, so that it can access directly repositories on GitHub.
  

  • Select activity Announcements

    

    Announcements Forum
  • Select activity Syllabus

    

    Syllabus URL
  • Select activity Discussion Forum

    

    Discussion Forum
  • Select activity Projects

    

    Projects Page
  • Select activity Live Poll

    

    Live Poll Choice
  • Select activity Homework 1

    

    Homework 1 Assignment

    Opened: Tuesday, 7 March 2023, 12:00 AM

    Due: Tuesday, 14 March 2023, 12:00 AM
• Select section Lectures

  Lectures

  Date	Topic	Material
  20/2/2023	Introduction	slides (1-36)
  21/2/2023	Introduction	slides (37-91)
  22/2/2023	Overview	slides
  27/2/2023	Language Models	slides Suggested readings: SLP, Chapter 3. Notebook: Language Models
  28/2/2023	Word Vectors	slides (1-24) Notebook: WordNet
  1/3/2023	Word Vectors	slides (25-53) video Suggested readings: Word2Vec Tutorial Efficient Estimation of Word Representations in Vector Space
  6/3/2023	Word2Vec Tutorial Contextual Word Embeddings	slides (54-86) Notebook: Word2Vec Tutorial WordEmbeddings
  7/3/2023	Text Classification	slides Suggested readings: SLP Chapter 6
  8/3/2023	Classification	slides Notebook: Neural classifier
  13/3/2023	Tokenization	slides Notebook: regular expressions
  14/3/2023	Hidden Markov Models	slides Suggested readings: SLP Chapter 9
  15/3/2023	Correction of Homework 1	Notebook: Homework1 solution
  20/3/2023	Named Entity Recognition	slides Notebook: Pos Tagger Suggested readings: SLP Chapter 9
  21/3/2023	Convolutional Neural Networks	slides Notebook: CNN for NLP
  22/3/2023	Recurrent Neural Networks for NLP	slides Notebook: RNN Tagger Further readings: Recurrent Neural Networks Tutorial The Unreasonable Effectiveness of Recurrent Neural Networks Sequence Modeling: Recurrent and Recursive Neural Nets
  27/3/2023	Long Short Term Memories	slides Further readings: Understanding LSTM Networks
  28/3/2023	Attention and the Transformer	slides (1-58) Video:
  29/3/2023	Transformer (Continued)	slides (59-77)
  3/4/2023	Transformer Architectures	slides Notebooks: Explore Transformers Sentence Classification with BERT
  12/4/2023	Prompt Tuning	slides Video:
  17/4/2023	ChatGPT and Reinforcement Learning	slides
  18/4/2023	Introduction to Python and Keras	Notebook: Into to Python
  19/4/2023	Introduction to TensorFlow	slides Notebook: Tensorflow Tutorial
  24/4/2023	Introduction to PyTorch. HuggingFace Tutorial	Notebooks: PyTorch HuggingFace
  26/4/2023	Parsing	slides (1-70) Suggested readings: SLP Chapters 12, 13, 14 Further readings: Joakim Nivre. 2004. Incrementality in Deterministic Dependency Parsing. Workshop on Incremental Parsing. Danqi Chen and Christopher D. Manning. 2014. A Fast and Accurate Dependency Parser using Neural Networks. EMNLP 2014. Sandra Kübler, Ryan McDonald, Joakim Nivre. 2009. Dependency Parsing. Morgan and Claypool. Daniel Andor, Chris Alberti, David Weiss, Aliaksei Severyn, Alessandro Presta, Kuzman Ganchev, Slav Petrov, and Michael Collins. 2016. Globally Normalized Transition-Based Neural Networks. ACL 2016.
  2/5/2023	Dependency Parsing: Training	slides (71-) Notebooks: DiaParser Neural Dependency Parsing
  3/5/2023	Analysis of Language Models (aka BERTology)	slides Notebooks: 02-transformers.ipynb 03-pipeline.ipynb
  8/5/2023	Machine Translation	slides
  9/5/2023	Statistical Machine Translation	slides
  10/5/2023	Neural Machine Translation	slides
  15/5/2022	Project: topics and advice	Project Topics Project Advice
  16/5/2023	Reading Comprehension	slides Further readings: SQuAD: 100,000+ Questions for Machine Comprehension of Text Bidirectional Attention Flow for Machine Comprehension Reading Wikipedia to Answer Open-Domain Questions
  17/5/2023	Open Domain Question Answering	slides Further readings: SQuAD: 100,000+ Questions for Machine Comprehension of Text Bidirectional Attention Flow for Machine Comprehension Reading Wikipedia to Answer Open-Domain Questions Latent Retrieval for Weakly Supervised Open Domain Question Answering Dense Passage Retrieval for Open-Domain Question Answering Learning Dense Representations of Phrases at Scale
  22/5/2023	Trends and Future of NLP	slides
  23/5/2023	Bonus: Thinking Fast and Slow	slides Further readings: D. Kahneman. Thinking Fast and Slow. Farrar, Straus and Giroux. 2013.
• Select section Textbooks

  Textbooks

  1. D. Jurafsky, J.H. Martin, Speech and Language Processing. 3rd edition, Prentice-Hall, 2021.
  2. S. Bird, E. Klein, E. Loper. Natural Language Processing with Python.

  3. Ian Goodfellow and Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016.

  Additional Material

  1. B. Liu, Sentiment Analysis and Opinion Mining. Morgan & Claypool Publishers, 2012.
     
  2. S. Kubler, R. McDonald, J. Nivre. Dependency Parsing. 2010.
     
  3. P. Koehn. Statistical Machine Translation. Cambridge University Press, 2010.
     

  
  
  

  
  
• Select section Tutorials

  Tutorials

  • A free online course on Python for Machine Learning.
• Select section Similar Courses

  Similar Courses

  • Deep Learning for NLP, Stanford, CS224n
  • Advanced NLP, CMU, CS 11-711, Fall 2021
• Select section Previous Editions

  Previous Editions

  • HLT 2022
  • HLT 2021
  • HLT 2020
    
  • HLT 2019
  • HLT 2018