Description
Survey of recent research advances in intelligent decision making for dynamic environments from a computational perspective. Efficient algorithms for single and multiagent planning in situations where a model of the environment may or may not be known. Partially observable Markov decision processes, approximate dynamic programming, and reinforcement learning. New approaches for overcoming challenges in generalization from experience, exploration of the environment, and model representation so that these methods can scale to real problems in a variety of domains including aerospace, air traffic control, and robotics. Students are expected to produce an original research paper on a relevant topic. Prerequisites: AA228/CS238 or CS221.
Lectures
Normal lectures will be Tuesdays and Thursdays from 9am-10:20am in McMurtry Art Building Oshman. On Tuesday Jan 30, lecture will take place in Building 200/034.
Learning Outcomes
As part of this course, students will:
1. Obtain a broad fundamental understanding of models and algorithms for sequential decision making (assessed by midterms).
2. Become familiar with the open research questions, pose new research questions, and write a proposal (assessed by project proposal).
3. Read technical papers, design and present a tutorial (assessed by course presentation) and a basic implementation in code.
4. Present original research, write a research paper, and critique other research papers (assessed by course project).
Units
AA229 will be offered for 3-4 units for either a letter grade or credit/no credit. Students registering for the 4 unit version of the course will be required to spend at least 30 additional hours extending their course project and preparing the paper for a peer-reviewed conference submission (actual submission is not required).
Textbook
This course will use papers that are available for free through Stanford’s subscription. Links to the PDFs are provided in the course schedule. If off campus, you will need to use http://pac.stanford.edu/suproxy.pac as your proxy.
Discussion
Class discussions are held on Ed.
Office Hours
Office hours are by appointment and can be scheduled by emailing arec(at)stanford.edu or mykel(at)stanford.edu.
Grading
Presentation: 35%
- 5% Draft Presentation (due at 5pm three days prior to presentation)
- 15% Presentation (due one hour prior to class)
- 10% Implementation Writeup (due Friday at 5pm in the week of the presentation)
- 5% Exercises (due Friday at 5pm in the week of the presentation)
Quizzes: 20%
- 10% Quiz 1
- 10% Quiz 2
Final project: 40%
- 5% Proposal
- 15% Presentation
- 15% Paper
- 5% Peer Review
Class participation: 5%
- 2% Attendance
- 3% Participation in discussion
Quizzes
Quizzes will be released at 5:00 pm on the Wednesdays indicated on the schedule and will be due at 5:00 pm on the Sunday of that week . You will submit your quiz as a PDF using Canvas. You may use any resources (such as books, software, webpages), but you may not consult with others (in the class or out) about the exam. Your quiz may be typed or handwritten and scanned. If any clarification is required, please send a private message to the course staff over Ed. In order to do well on the exam, you will need to attend class, keep up with the associated readings, and do the practice exercises. A student who is well studied should be able to complete the exam in 90 minutes.
Students with Disabilities
Students who may need an academic accommodation based on the impact of a disability must initiate the request with the Office of Accessible Education (OAE). Professional staff will evaluate the request with required documentation, recommend reasonable accommodations, and prepare an Accommodation Letter for faculty dated in the current quarter in which the request is made. Students should contact the OAE as soon as possible since timely notice is needed to coordinate accommodations. The OAE is located at 563 Salvatierra Walk (phone: 723-1066, URL: http://studentaffairs.stanford.edu/oae).
