MS&E 235A / EE 283: Markov Decision Processes

Lectures

Location: 540-108, Mon Wed 3:00 PM - 4:20 PM

Instructor Office Hours

Professor Benjamin Van Roy: Fri 3:00 PM - 4:00 PM, Packard 273.

Benlin: Thu 1:00 PM - 2:30 PM, Packard 230.

Course Requirements

Homework: There will be approximately weekly homework sets. Homework assignments will be posted on Canvas, and are due by 11:59 PM on Fridays on gradescope.

Midterm: October 29 (Wed), 3:00 PM - 4:20 PM at 540-108

Final: December 8 (Tue), 8:30 AM - 11:30 AM at 540-108.

More details will be given at the introductory lecture.

Grading

Homework 25%, midterm 25%, final 50%

AI Policy

Feel free to use AI for anything except the exams.

Course Prerequisites

The course will rely on knowledge of probability and programming. Requisite background in probability can be obtained through MS&E 121, EE 178, or CS 109, and for programming, CS106B. Students are expected, for example, to be proficient in working with Gaussian, exponential, and beta distributions, Markov chains, and notions of convergence. Students are expected to also have experience in developing and debugging software implementations of numerical algorithms.

This course satisfies requirements for the Data and Decisions and the Computational Social Sciences programs in MS&E and for the Information Systems and Science program in EE.