MS&E 235A / EE 283: Markov Decision Processes

Lectures

Location: 60-109, Mon Wed 3:00 PM - 4:20 PM

Instructor Office Hours

Professor Benjamin Van Roy: Fri 3:00 PM - 4:00 PM, Packard 273.

Yifan: Thu 4:00 PM - 5:30 PM, Packard 107.

Course Requirements

Homework: There will be approximately weekly homework sets. Homework assignments are due by 11:59 PM on Fridays.

Midterm: October 29 (Wed), 3:00 PM - 4:20 PM at 60-109

Final: December 9 (Tue), 8:30 AM - 11:30 AM at TBD.

More details will be given at the introductory lecture.

Grading

Homework 25%, midterm 25%, final 50%

Course Prerequisites

The course will rely on knowledge of probability and programming. Requisite background in probability can be obtained through MS&E 121, EE 178, or CS 109, and for programming, CS106B. Students are expected, for example, to be proficient in working with Gaussian, exponential, and beta distributions, Markov chains, and notions of convergence. Students are expected to also have experience in developing and debugging software implementations of numerical algorithms.

This course satisfies requirements for the Data and Decisions and the Computational Social Sciences programs in MS&E and for the Information Systems and Science program in EE.