Description

This course introduces decision making under uncertainty from a computational perspective and provides an overview of the necessary tools for building autonomous and decision-support systems. Following an introduction to probabilistic models and decision theory, the course will cover computational methods for solving decision problems with stochastic dynamics, model uncertainty, and imperfect state information. Topics include Bayesian networks, influence diagrams, dynamic programming, reinforcement learning, and partially observable Markov decision processes. Applications cover air traffic control, aviation surveillance systems, autonomous vehicles, and robotic planetary exploration.

Prerequisites: basic probability and fluency in a high-level programming language.

Lectures

Lectures will be Tuesdays and Thursdays from 1:30pm to 2:50pm in NVidia Auditorium.

Optional problem sessions from 3:30-4:20pm in Gates B01. These sessions will be recorded as well. Bring your questions.

The course is also available to the public through the Stanford Center for Professional Development (apply). Stanford students can watch videos here. Registered students  Taking another course that is offered at the same time is not an issue.

Major Themes

  • There is a principled mathematical framework for defining rational behavior.
  • Key algorithms for solving decision problems leverage decomposition and recursion.
  • The computational techniques discussed in class can lead to superior decisions that are sometimes counterintuitive.
  • Successful application of these principles depends on the choice of representation and approximation.
  • The same computational approaches can be applied to very different application domains.

Learning Outcomes

  • You will gain a broad fundamental understanding of the mathematical models and solution methods for decision making (exercises, two midterms, take-home quiz).
  • You will be able to implement and extend key algorithms for learning and decision making (two programming projects).
  • You will be able to identify an application of the theory in this course and formulate it mathematically (proposal).
  • You will gain a deep understanding of an area of particular interest and apply it to a problem (final project).
  • You will be able to critique approaches to solving decision problems (peer review).

Units

AA228 will be offered for 3-4 units. Students registering for the 4 unit version of the course will be required to spend at least 30 additional hours extending their course project and preparing the paper for a peer-reviewed conference submission (actual submission is not required).

Textbook

Mykel J. Kochenderfer, Decision Making Under Uncertainty: Theory and Application, MIT Press, 2015. $70.00

Stanford students can access the online copy here.

Errata for the first printing can be found here.

Discussion

Class discussions are held on Piazza.

Grading

Projects: 31%

  • 1% Project 0
  • 15% Project 1
  • 15% Project 2

Midterms: 44%

  • 14% Midterm 1
  • 14% Midterm 2
  • 16% Midterm 3

Final project: 25%

  • 5% Proposal
  • 5% Status Update
  • 10% Paper
  • 5% Peer Review

Grades are posted on Canvas.

Late Policy

The Late Policy is a 20% penalty per day. For the paper and peer review, there is a 20% penalty per hour (since we need to distribute the papers for peer review, and the peer reviews are required to submit the final grade).

Exams

Dates

See Schedule.

Conflicts

If you are not able to attend the in class midterms with an official reason, please let the course staff know ASAP so that a separate examination time can be scheduled.

Notes

Students are allowed to bring only 1 double sided page (letter size) of typed or handwritten notes to the midterm. For the second midterm, you may bring the same sheet your brought to your first midterm along as well.

SCPD Students

We will be flexible with the examination times let you choose anytime on the day of the midterm exam to do it. You can arrange the exact time with your monitor.

Students with Disabilities

Students who may need an academic accommodation based on the impact of a disability must initiate the request with the Office of Accessible Education (OAE). Professional staff will evaluate the request with required documentation, recommend reasonable accommodations, and prepare an Accommodation Letter for faculty dated in the current quarter in which the request is made. Students should contact the OAE as soon as possible since timely notice is needed to coordinate accommodations. The OAE is located at 563 Salvatierra Walk (phone: 723-1066, URL: http://studentaffairs.stanford.edu/oae).