Calendar

Mon	Tue	Wed	Thu	Fri
Sep 23 Lecture 1: Introduction	Sep 24	Sep 25 PA1 out Lecture 2: Word Alignment Models for Statistical MT	Sep 26	Sep 27
Sep 30 Lecture 3: Machine Translation: Word Alignment, Parallel Corpora, Decoding, Evaluation	Oct 1	Oct 2 Lecture 4: Modern MT Systems (Phrase-based, Syntactic)	Oct 3	Oct 4
Oct 7 Lecture 5: N-Grams, Final Project Discussion	Oct 8	Oct 9 PA1 due Lecture 6: Syntax and parsing PA2 out	Oct 10	Oct 11
Oct 14 Lecture 7: Competitive Grammar Writing I	Oct 15	Oct 16 Lecture 8: Competitive Grammar Writing II	Oct 17	Oct 18
Oct 21 Final project proposal due Lecture 9: Dependency Parsing	Oct 22	Oct 23 PA2 due Lecture 10: Coreference Resolution PA3 out	Oct 24	Oct 25
Oct 28 Lecture 11: Coreference Resolution II	Oct 29	Oct 30 Lecture 12: MaxEnt Classifiers, Sequence Classifiers for NER	Oct 31	Nov 1
Nov 4 Lecture 13: MaxEnt Classifiers, Sequence Classifiers for NER	Nov 5	Nov 6 PA3 due Lecture 14: Deep Learning for NLP PA4 out	Nov 7	Nov 8
Nov 11 Lecture 15: Deep Learning for Sequence Models	Nov 12	Nov 13 Lecture 16: Topic Models	Nov 14	Nov 15
Nov 18 Lecture 17: Lexical Semantics	Nov 19	Nov 20 Lecture 18: Computational Semantics	Nov 21	Nov 22
Nov 25 Thanksgiving	Nov 26	Nov 27 Thanksgiving	Nov 28	Nov 29
Dec 2 Lecture 19: Computational Semantics (continued)	Dec 3	Dec 4 Lecture 20: Text-based Question Answering systems	Dec 5	Dec 6 Final project / PA4 due
Dec 9	Dec 10 Final project presentations	Dec 11	Dec 12	Dec 13

Syllabus

Lecture 1
Mon
9/23/13

Course Introduction and Administration. Overview of NLP. Statistical Machine Translation.

Lecture Slides: (1-up) (6-up)

Required:

If your knowledge of probability theory is limited, please read M&S 2.0-2.1.7. If that's too condensed, read the probability chapter of an intro statistics textbook, e.g. Rice, Mathematical Statistics and Data Analysis, ch. 1.

Optional:

Good background reading: J&M Ch. 1; M&S 1.0-1.3, 4.1-4.2, Collaboration Policy
Unix text manipulation (useful skill!): Ken Church's tutorial Unix for Poets or the (more up-to-date!) cs124 version.
Background for MT video [fun read!]: The IBM 701 translator (1954)

Lecture 2
Wed
9/25/13

Word Alignment Models for Statistical MT

Assignments:

PA1 (Word Alignment and MT System) Out. (Find it on OpenEdX under Courseware.)

Lecture Slides: (1-up) (6-up)

Tutorial reading:

Collins, Michael. 2011. Statistical Machine Translation: IBM Models 1 and 2.
Kevin Knight. A Statistical MT Tutorial Workbook. MS., August 1999. Sections 1-14.

Optional:

Moore, Robert C. 2005. Association-Based Bilingual Word Alignment. In Proceedings, Workshop on Building and Using Parallel Texts: Data-Driven Machine Translation and Beyond, Ann Arbor, Michigan , pp. 1-8.
Moore, Robert C. 2004. Improving IBM Word Alignment Model 1. In Proceedings, 42nd Annual Meeting of the Association for Computational Linguistics, Barcelona, Spain, pp. 519-526.
Dyer, Chris, Victor Chahuneau and Noah A. Smith. 2013. A Simple, Fast, and Effective Reparameterization of IBM Model 2. Proceedings of NAACL 2013.

Lecture 3
Mon
9/30/13

Machine Translation: Word Alignment, Parallel Corpora, Decoding, Evaluation

Lecture Slides: (1-up) (6-up)

Required:

J&M chapter 25

Tutorial reading:

Kevin Knight. A Statistical MT Tutorial Workbook. MS., August 1999. Sections 15-37.
Knight workbook FAQ

Optional:

Ulrich Germann, Michael Jahr, Kevin Knight, Daniel Marcu, and Kenji Yamada. 2001. Fast Decoding and Optimal Decoding for Machine Translation. ACL.
Adam Lopez. 2008. Statistical Machine Translation. ACM Computing Surveys 40(3).

Lecture 4
Wed
10/2/13

Modern MT Systems (Phrase-based, Syntactic)

Lecture Slides: (1-up) (6-up)

Optional:

Franz Josef Och, Hermann Ney. 2004. The alignment template approach to statistical machine translation. Computational Linguistics 30(4): 417-449.
K. Yamada and K. Knight. 2002. A Decoder for Syntax-Based Statistical MT. ACL.
David Chiang. 2005. A hierarchical phrase-based model for statistical machine translation. ACL 2005, pages 263-270.

Lecture 5
Mon
10/7/13

N-Grams, Final Project Discussion

Lecture Slides: (1-up) (6-up)

Required:

J&M chapter 4 (or M&S 1.4, 2.2, 6)
Week 3 Language Modeling Videos

Resources:

MegaHAL
Shannon experiment for the entropy of English: Java applet. Results spreadsheet: gdoc

Optional:

(advanced) Joshua Goodman (2001), A Bit of Progress in Language Modeling, Extended Version.
(advanced) Stanley Chen and Joshua Goodman (1998), An empirical study of smoothing techniques for language modeling.
(very advanced) Teh, Yee Whye. 2006. A Hierarchical Bayesian Language Model based on Pitman-Yor Processes. EMNLP 2006.

Lecture 6
Wed
10/9/13

Syntax and parsing

Lecture Slides: (1-up) (6-up)

Assignments:

PA1 due
PA2 (CYK-Parser) out

Required:

Week 3 Parsing Videos
J&M ch. 13, secs. 13.0-13.3.

Background:

J&M ch. 9 (or M&S ch. 3). This is especially if you haven't done any linguistics courses, but even if you have, there's useful information on treebanks and part-of-speech tag sets used in NLP.

Lecture 7
Mon
10/14/13

Competitive Grammar Writing I

Required:

Week 4 Parsing Videos
J&M sec 13.4

Background:

Dan Klein and Christopher D. Manning. 2003. Accurate Unlexicalized Parsing. ACL 2003, pp. 423-430.

Lecture 8
Wed
10/16/13

Competitive Grammar Writing II

Required:

Week 4 Parsing Videos

Optional:

Eugene Charniak (1997), Statistical techniques for natural language parsing, AI Magazine.
Eugene Charniak (1997), Statistical parsing with a context-free grammar and word statistics, Proceedings of the Fourteenth National Conference on Artificial Intelligence. AAAI Press/MIT Press, Menlo Park (1997).
Eugene Charniak (2000), A Maximum-Entropy-Inspired Parser, Proceedings of NAACL-2000.
Slav Petrov, Leon Barrett, Romain Thibaux and Dan Klein. Learning Accurate, Compact, and Interpretable Tree Annotation. COLING/ACL 2006

Lecture 9
Mon
10/21/13

Dependency Parsing

Lecture Slides: (1-up) (6-up)

Assignments:

Final Project Proposal Due

Required:

Week 5 Parsing Videos

Lecture 10
Wed
10/23/13

Coreference Resolution

Lecture Slides: (1-up) (6-up)

Assignments:

PA2 Due
PA3 (Coreference System) out

Required:

J&M 21.3-21.8 (or all of Chapter 21 if you wish!)

Optional:

Ng, Vincent (2005). Supervised Noun Phrase Coreference Research: The First Fifteen Years.
Raghunathan, Karthik et al. (2010). A Multi-Pass Sieve for Coreference Resolution. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing (EMNLP 2010).

Lecture 11
Mon
10/28/13

Coreference Resolution II

Lecture Slides: (1-up) (6-up) Intro to feature-based classifiers: (1-up) (6-up)

Lecture 12
Wed
10/30/13

MaxEnt Classifiers, Sequence Classifiers for NER

Lecture Slides: (1-up) (6-up)

Optional:

J&M sections 6.6-6.8, 22.1
Adam Berger, A Brief Maxent Tutorial

Lecture 13
Mon
11/4/13

MaxEnt Classifiers, Sequence Classifiers for NER

Lecture Slides: (1-up) (6-up)

Lecture 14
Wed
11/6/13

Deep Learning for NLP

Lecture Slides: (1-up) (6-up)

Assignments:

PA3 Due
PA4 (Deep Learning Sequence Model) out

Lecture 15
Wed
11/11/13

Deep Learning for Sequence Models

Lecture Slides: (1-up) (6-up)

Lecture 16
Wed
11/13/13

Topic Models

Lecture Slides: (1-up) (Dissertation Browser)

Lecture 17
Mon
11/18/13

Lexical Semantics

Lecture Slides: (1-up)

Lecture 18
Wed
11/20/13

Computational Semantics

Lecture Slides: (1-up) (6-up)

Thanksgiving Break

Lecture 19
Mon
12/2/13

Computational Semantics (continued)

Lecture Slides: first part (1-up) (6-up); second part (1-up); third part (1-up).

Required:

An Introduction to Formal Computational Semantics

Lecture 20
Wed
12/4/13

Text-based Question Answering systems

Lecture Slides: (1-up)

Required:

J&M 23.0, 23.2

Optional:

Marius Pasca, Sanda M. Harabagiu. High Performance Question/Answering. SIGIR 2001: 366-374.
Ferrucci et al. Building Watson: An Overview of the DeepQA Project. AI Magazine, 2010.

Tue
12/10/13 3:30-6:00pm
Gates B01

Final Project Presentations