Calendar

Mon	Tue	Wed	Thu	Fri
Sep 22 Lecture 1: Introduction	Sep 23	Sep 24 Lecture 2: Word Alignment Models for Statistical MT	Sep 25 PA1 out	Sep 26
Sep 29 Lecture 3: Machine Translation: Word Alignment, Parallel Corpora, Decoding, Evaluation	Sept 30	Sept 31 Lecture 4: Modern MT Systems (Phrase-based, Syntactic)	Oct 2	Oct 3
Oct 6 Lecture 5: N-Grams, Final Project Discussion	Oct 7	Oct 8 Lecture 6: Syntax and parsing PA2 out	Oct 9 PA1 due	Oct 10
Oct 13 Lecture 7: Competitive Grammar Writing I	Oct 14	Oct 15 Lecture 8: Competitive Grammar Writing II	Oct 16	Oct 17
Oct 20 Final project proposal due Lecture 9: Dependency Parsing	Oct 21	Oct 22 PA2 due Lecture 10: Coreference Resolution PA3 out	Oct 23	Oct 24
Oct 27 Lecture 11: Coreference Resolution II / Classifiers	Oct 28	Oct 29 Lecture 12: MaxEnt Classifiers, Sequence Classifiers for NER	Oct 30	Oct 31
Nov 3 Lecture 13: Kevin Knight's Lecture: Language Translation and Code-Breaking	Nov 4	Nov 5 PA3 due Lecture 14: Deep Learning for NLP PA4 out	Nov 6	Nov 7
Nov 10 Lecture 15: Deep Learning for NLP	Nov 11	Nov 12 Lecture 16: Deep Learning for NLP	Nov 13	Nov 14
Nov 17 Lecture 17: Computational Semantics	Nov 18	Nov 19 Lecture 18: Computational Semantics	Nov 20	Nov 21
Nov 24 Thanksgiving	Nov 25	Nov 26 Thanksgiving	Nov 27	Nov 28
Dec 1 Lecture 19: Computational Semantics (continued)	Dec 2	Dec 3 Lecture 20: Text-based Question Answering systems	Dec 4	Dec 5 Final project / PA4 due
Dec 8 Hard deadline for final project report	Dec 9	Dec 10 Final project presentations 3:30-6:30pm	Dec 11	Dec 12

Syllabus

Lecture 1
Mon
9/22/14

Course Introduction and Administration. Overview of NLP. Statistical Machine Translation.

Lecture Slides: (1-up) (6-up)

Required:

If your knowledge of probability theory is limited, please read M&S 2.0-2.1.7. If that's too condensed, read the probability chapter of an intro statistics textbook, e.g. Rice, Mathematical Statistics and Data Analysis, ch. 1.

Optional:

Good background reading: J&M Ch. 1; M&S 1.0-1.3, 4.1-4.2, Collaboration Policy
Unix text manipulation (useful skill!): Ken Church's tutorial Unix for Poets or the (more up-to-date!) cs124 version.

Lecture 2
Wed
9/24/14

Word Alignment Models for Statistical MT

Assignments:

PA1 (Word Alignment and MT System) Out. (Find it on OpenEdX under Courseware.)

Lecture Slides: (1-up) (6-up)

Tutorial reading:

Collins, Michael. 2011. Statistical Machine Translation: IBM Models 1 and 2.
Kevin Knight. A Statistical MT Tutorial Workbook. MS., August 1999. Sections 1-14.

Optional:

Moore, Robert C. 2005. Association-Based Bilingual Word Alignment. In Proceedings, Workshop on Building and Using Parallel Texts: Data-Driven Machine Translation and Beyond, Ann Arbor, Michigan , pp. 1-8.
Moore, Robert C. 2004. Improving IBM Word Alignment Model 1. In Proceedings, 42nd Annual Meeting of the Association for Computational Linguistics, Barcelona, Spain, pp. 519-526.
Dyer, Chris, Victor Chahuneau and Noah A. Smith. 2013. A Simple, Fast, and Effective Reparameterization of IBM Model 2. Proceedings of NAACL 2013.
Background for MT video [fun read!]: The IBM 701 translator (1954)

Lecture 3
Mon
9/29/14

Machine Translation: Word Alignment, Parallel Corpora, Decoding, Evaluation

Lecture Slides: (1-up) (6-up)

Required:

J&M chapter 25

Tutorial reading:

Kevin Knight. A Statistical MT Tutorial Workbook. MS., August 1999. Sections 15-37.
Knight workbook FAQ

Optional:

Ulrich Germann, Michael Jahr, Kevin Knight, Daniel Marcu, and Kenji Yamada. 2001. Fast Decoding and Optimal Decoding for Machine Translation. ACL.
Adam Lopez. 2008. Statistical Machine Translation. ACM Computing Surveys 40(3).

Lecture 4
Wed
10/1/14

Modern MT Systems (Phrase-based, Syntactic)

Lecture Slides: (1-up) (6-up)

Optional:

Franz Josef Och, Hermann Ney. 2004. The alignment template approach to statistical machine translation. Computational Linguistics 30(4): 417-449.
K. Yamada and K. Knight. 2002. A Decoder for Syntax-Based Statistical MT. ACL.
David Chiang. 2005. A hierarchical phrase-based model for statistical machine translation. ACL 2005, pages 263-270.

Lecture 5
Mon
10/6/14

N-Grams, Final Project Discussion

Lecture Slides: (1-up) (6-up)

Required:

J&M Chapter 4 (EdX link) (or M&S 1.4, 2.2, 6)
Week 3 Language Modeling Videos

Resources:

MegaHAL
Shannon experiment for the entropy of English: Java applet. Results spreadsheet: gdoc

Optional:

(advanced) Joshua Goodman (2001), A Bit of Progress in Language Modeling, Extended Version.
(advanced) Stanley Chen and Joshua Goodman (1998), An empirical study of smoothing techniques for language modeling.
(very advanced) Teh, Yee Whye. 2006. A Hierarchical Bayesian Language Model based on Pitman-Yor Processes. EMNLP 2006.

Lecture 6
Wed
10/8/14

Syntax and parsing

Lecture Slides: (1-up) (6-up)

Assignments:

PA1 due
PA2 (CYK-Parser) out

Required:

Week 3 Parsing Videos
J&M ch. 13, secs. 13.0-13.3.

Background:

J&M ch. 9 (or M&S ch. 3). This is especially if you haven't done any linguistics courses, but even if you have, there's useful information on treebanks and part-of-speech tag sets used in NLP.

Lecture 7
Mon
10/13/14

Competitive Grammar Writing I

Required:

Week 4 Parsing Videos
J&M sec 13.4

Background:

Dan Klein and Christopher D. Manning. 2003. Accurate Unlexicalized Parsing. ACL 2003, pp. 423-430.

Lecture 8
Wed
10/15/14

Competitive Grammar Writing II

Required:

Week 4 Parsing Videos

Optional:

Eugene Charniak (1997), Statistical techniques for natural language parsing, AI Magazine.
Eugene Charniak (1997), Statistical parsing with a context-free grammar and word statistics, Proceedings of the Fourteenth National Conference on Artificial Intelligence. AAAI Press/MIT Press, Menlo Park (1997).
Eugene Charniak (2000), A Maximum-Entropy-Inspired Parser, Proceedings of NAACL-2000.
Slav Petrov, Leon Barrett, Romain Thibaux and Dan Klein. Learning Accurate, Compact, and Interpretable Tree Annotation. COLING/ACL 2006