Schedule and Syllabus

Unless otherwise specified the course lectures and meeting times are:

Tuesday, Thursday 4:30-5:50
Location: NVIDIA Auditorium
EventDateDescriptionCourse Materials
Lecture Jan 9 Introduction to NLP and Deep Learning
[slides]		 Suggested Readings:
  1. [Linear Algebra Review]
  2. [Probability Review]
  3. [Convex Optimization Review]
  4. [More Optimization (SGD) Review]
Lecture Jan 11 Word Vectors 1
[slides] 		Suggested Readings:
  1. [Word2Vec Tutorial - The Skip-Gram Model]
  2. [Distributed Representations of Words and Phrases and their Compositionality]
  3. [Efficient Estimation of Word Representations in Vector Space]
A1 released Jan 11 Assignment #1 released [Assignment #1][ Written Solutions ]
Lecture Jan 16 Word Vectors 2
[slides]		 Suggested Readings:
  1. [GloVe: Global Vectors for Word Representation]
  2. [Improving Distributional Similarity with Lessons Learned fromWord Embeddings]
  3. [Evaluation methods for unsupervised word embeddings]
Lecture Jan 18 Neural Networks
[slides]		 Suggested Readings:
  1. cs231n notes on [backprop] and [network architectures]
  2. [Review of differential calculus]
  3. [Natural Language Processing (almost) from Scratch]
  4. [Learning Representations by Backpropagating Errors]
Review Jan 19 Python Refresher [ slides ]
Lecture Jan 23 Backpropagation and Project Advice
[slides] [lecture notes]		 Suggested Readings:
  1. [Derivatives, Backpropagation, and Vectorization]
  2. [Yes you should understand backprop]
Lecture Jan 25 Introduction to TensorFlow
[slides] [lecture code] 		Suggested Readings:
  1. [TensorFlow Basic Usage]
A1 Due Jan 25 Assignment #1 due
A2 Released Jan 25 Assignment #2 released [Assignment #2] [ Written Solutions ]
Lecture Jan 30 Dependency Parsing
[slides]		 Suggested Readings:
  1. Joakim Nivre. 2004. Incrementality in Deterministic Dependency Parsing. Workshop on Incremental Parsing.
  2. Danqi Chen and Christopher D. Manning. 2014. A Fast and Accurate Dependency Parser using Neural Networks. EMNLP 2014.
  3. Sandra Kübler, Ryan McDonald, Joakim Nivre. 2009. Dependency Parsing. Morgan and Claypool. [Free access from Stanford campus, only!]
  4. Daniel Andor, Chris Alberti, David Weiss, Aliaksei Severyn, Alessandro Presta, Kuzman Ganchev, Slav Petrov, and Michael Collins. 2016. Globally Normalized Transition-Based Neural Networks. ACL 2016.
  5. Marie-Catherine de Marneffe, Timothy Dozat, Natalia Silveira, Katri Haverinen, Filip Ginter, Joakim Nivre, and Christopher D. Manning. 2014. Universal Stanford Dependencies: A cross-linguistic typology. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014). Revised version for UD v1.
  6. Universal Dependencies website
Lecture Feb 1 Recurrent Neural Networks and Language Models
[slides]		 Suggested Readings:
[N-gram Language Models and Perplexity]
[The Unreasonable Effectiveness of Recurrent Neural Networks]
[Recurrent Neural Networks Tutorial]
[Sequence Modeling: Recurrent and Recursive Neural Nets]
DFP Released Feb 1 Default Final Project released
Lecture Feb 6 Vanishing Gradients, Fancy RNNs
[slides] 		Suggested Readings:
[Understanding LSTM Networks]
[Vanishing Gradients Example]
Review Feb 8 Midterm Review
[slides] 		This year's midterm will be most similar to practice midterm 3 (the first two are from cs224d).
[practice midterm 1] [with solutions]
[practice midterm 2] [with solutions]
[practice midterm 3] [with solutions]
Project Proposal Due Feb 8 Final Project proposal due Final Project Proposal
A2 Due Feb 8 Assignment #2 due
Alternate Midterm Feb 9 Alternate Midterm
A3 Released Feb 13 Assignment #3 released Assignment #3 [ Written Solutions ]
Midterm Feb 13 In-class midterm Location: Memorial Auditorium, Time: 4:30 - 5:50pm
[Midterm]
[Midterm Solutions]
Lecture Feb 15 Machine Translation, Seq2Seq and Attention
[slides] 		Suggested Readings:
[Statistical Machine Translation slides (see lectures 2/3/4)]
[Statistical Machine Translation Book]
[BLEU metric]
[Original sequence-to-sequence NMT paper (also describes beam search)]
[Earlier sequence-to-sequence speech recognition paper (includes detailed beam search alg)]
[Original sequence-to-sequence + attention paper]
[Guide to attention and other RNN augmentations]
[Massive Exploration of Neural Machine Translation Architectures]
Lecture Feb 20 Advanced Attention
[slides]		 Suggested Readings:
[A Deep Reinforced Model for Abstractive Summarization]
[Get To The Point: Summarization with Pointer-Generator Networks]
[BlackOut: Speeding up Recurrent Neural Network Language Models with very Large Vocabularies]
[Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models]
[Quasi-Recurrent Neural Networks]
Lecture Feb 22 Transformer Networks and CNNs
[slides]		 Suggested Readings:
[Attention Is All You Need]
[Layer Normalization]
[Convolutional Neural Networks for Sentence Classification]
[Improving neural network3s by preventing co-adaptation of feature detectors]
[A Convolutional Neural Network for Modelling Sentences]
Lecture Feb 27 Coreference Resolution
[slides] 		Suggested Readings:
[Learning Anaphoricity and Antecedent Ranking Features for Coreference Resolution]
[Improving Coreference Resolution by Learning Entity-Level Distributed Representations]
[End-to-end Neural Coreference Resolution]
[Coreference Demo]
A3 Due Feb 27 Assignment #3 due
Milestone Due Feb 28 Final project milestone due Project Milestone
Lecture Mar 1 Tree Recursive Neural Networks and Constituency Parsing
[slides]
Lecture Mar 6 Advanced Architectures and Memory Networks
[slides]
Lecture Mar 8 Reinforcement Learning for NLP Guest Lecture
[slides]		 Suggested Readings:
[A Deep Reinforced Model for Abstractive Summarization]
[DCN+: Mixed Objective and Deep Residual Coattention for Question Answering]
[Deep Reinforcement Learning for Dialogue Generation]
Lecture Mar 13 Semi-supervised Learning for NLP
[slides] 		Suggested Readings:
[Semi-Supervised Sequence Learning]
[Learned in Translation: Contextualized Word Vectors]
[Deep Contextualized Word Representations]
[Adversarial Training Methods for Semi-Supervised Text Classification]
Lecture Mar 15 Future of NLP Models, Multi-task Learning and QA Systems
[slides]
Final Project Due Mar 18 Final project due
Poster Presentation Mar 21 Final project poster presentations
 5:30-8:30
McCaw Hall at the Alumni Center
McCaw Hall at the Alumni Center
More Details