AA228/CS238 will be offered starting September 2020. Due to COVID-19, it will be taught online. You do not have to attend lectures live; recordings will be made available through Canvas. Office hours will be in Nooks.

Description

This course introduces decision making under uncertainty from a computational perspective and provides an overview of the necessary tools for building autonomous and decision-support systems. Following an introduction to probabilistic models and decision theory, the course will cover computational methods for solving decision problems with stochastic dynamics, model uncertainty, and imperfect state information. Topics include Bayesian networks, influence diagrams, dynamic programming, reinforcement learning, and partially observable Markov decision processes. Applications cover air traffic control, aviation surveillance systems, autonomous vehicles, and robotic planetary exploration.

Prerequisites: basic probability and fluency in a high-level programming language.

Lectures

Lectures will be by Zoom Tuesdays and Thursdays, 2:30pm to 3:50pm. Zoom links will be provided through Canvas. Recorded videos will be made available for offline viewing within one hour after the end of lecture.

The course is also available to the public through the Stanford Center for Professional Development (apply). Registered students taking another course that is offered at the same time is not an issue.

The draft textbook serves as the official lecture notes. Please note that those sketches are not a complete representation of all of the topics we discuss in class.

Major Themes

  • There is a principled mathematical framework for defining rational behavior.
  • Key algorithms for solving decision problems leverage decomposition and recursion.
  • The computational techniques discussed in class can lead to superior decisions that are sometimes counterintuitive.
  • Successful application of these principles depends on the choice of representation and approximation.
  • The same computational approaches can be applied to very different application domains.

Learning Outcomes

  • You will gain a broad fundamental understanding of the mathematical models and solution methods for decision making (exercises, two midterms, take-home quiz).
  • You will be able to implement and extend key algorithms for learning and decision making (two programming projects).
  • You will be able to identify an application of the theory in this course and formulate it mathematically (proposal).
  • You will gain a deep understanding of an area of particular interest and apply it to a problem (final project).
  • You will be able to critique approaches to solving decision problems (peer review).

Units

AA228 will be offered for 3 or 4 units for either a letter or credit/no credit grade. Students registering for the 4 unit version of the course will be required to spend at least 30 additional hours extending their course project and preparing the paper for a peer-reviewed conference submission (actual submission is not required). The grade breakdown listed in the “Grading” section is the same regardless of whether the class is taken for 3 or 4 units.

Textbook

This year we will be using a draft of a new textbook titled Algorithms for Decision Making. PDFs of the chapters will be made available as the course progresses. Feedback on the draft is welcome.

Prior years used a predecessor of this textbook, which can serve as an additional resource, although the new draft textbook is generally a superset of the material included: Mykel J. Kochenderfer, Decision Making Under Uncertainty: Theory and Application, MIT Press, 2015. $70.00. Stanford students can access the online copy here. Errata for earlier printings can be found here.

Discussion

Class discussions are held on Piazza.

Grading

Projects: 31%

  • 1% Project 0
  • 15% Project 1
  • 15% Project 2

Midterms: 45%

  • 0% Quiz 0
  • 15% Quiz 1
  • 15% Quiz 2
  • 15% Quiz 3

Final project: 24%

  • 5% Proposal
  • 0% Status Update (Optional)
  • 14% Paper or Video
  • 5% Peer Review

Grades are posted on Canvas.

Late Policy

The Late Policy is a 20% penalty per day. For the paper and peer review, there is a 20% penalty per hour (since we need to distribute the papers for peer review, and the peer reviews are required to submit the final grade). The quizzes have no late days.

Students with Disabilities

Students who may need an academic accommodation based on the impact of a disability must initiate the request with the Office of Accessible Education (OAE). Professional staff will evaluate the request with required documentation, recommend reasonable accommodations, and prepare an Accommodation Letter for faculty dated in the current quarter in which the request is made. Students should contact the OAE as soon as possible since timely notice is needed to coordinate accommodations. The OAE is located at 563 Salvatierra Walk (phone: 723-1066, URL: http://studentaffairs.stanford.edu/oae).