Schedule

Meeting Times And Location

Unless otherwise specified the course lectures and meeting times are:

Monday, Wednesday 11:30 AM - 12:50 PM
Location: Bishop Auditorium

Schedule And Course Materials

The preliminary schedule is given below and is subject to change. We will also be posting suggested readings in this section before each lecture. The lecture slides will be posted after each class, and sometimes before class. Please check back regularly for updates !

The lecture videos are recorded. You can watch them here.

In addition, in case it is helpful as an additional resource, we share past years' draft lexture notes for the class material: see more information here.
EventDate        DescriptionCourse Materials
Lecture Jan 6 Introduction to Reinforcement Learning
    -
  1. Slides
  2. Additional Materials:
Lecture Jan 8 Tabular MDP planning
  1. [Slides,Slides post class with annotations]
  2. Additional Materials:
    • SB (Sutton and Barto) Chp 3, 4.1-4.4
A1 Jan 8 Assignment 1 released
Lecture Jan 13 Tabular RL policy evaluation
  1. [Pre-class Slide Draft, Post class Slides with answerss]
  2. Additional Materials:
    3.
Lecture Jan 15 Q-learning
  1. [Slides, Class slides post class, Corrected slides, no annotations]
  2. Additional Materials:
    • SB (Sutton and Barto) Chp 5.2, 5.4, 6.4-6.5, 6.7
Jan 20 No Lecture
Lecture Jan 22 RL with function approximation
  1. [Slides, Class slides with annotations
  2. Additional Materials:
A1 Jan 22 Assignment 1 due, 11:59pm [Assignment 1]
-->
A2 Jan 22 Assignment 2 released
Lecture Jan 27 RL with function approximation
  1. [Slides, Class slides with annotations (released post class date)]
  2. Additional Materials:
Lecture Jan 29 RL with function approximation
  1. [Draft slides, Class slides with annotations (posted after class)]
Lecture Feb 3 Policy search
  1. [Draft slides, Class slides (post class)]
  2. Additional Materials:
    • SB (Sutton and Barto) Chp 13
Project Feb 3 Project proposal due, 11:59pm
Lecture Feb 5 Policy search
  1. [Draft slides, Class slides, posted after class
  2. Additional Materials
    • Sutton and Barto Chp 13
Project Feb 5 Assignment 2 due, 11:59pm
Exam Feb 10 In-class Midterm Past years have similar but not identical material. Prior midterms include: 2017 exam, 2017 solution, 2018 exam, 2018 solution, 2019 exam and 2019 solution
A3 Feb 10 Assignment 3 released
Lecture Feb 12 Exploration
  1. [Class slides with annotations]
  2. Additional Materials:
Lecture Feb 17 No Class: President's Day Holiday
Lecture Feb 19 Exploration/Exploitation
  1. [Draft slides, Class slides with annotations]
  2. Additional Materials:
Lecture Feb 24 Exploration / Exploitation
  1. [Class slides,post class slides, Draft lecture notes]
A3 Feb 24 Assignment 3 due, 11:59pm
Lecture Feb 26 Batch Reinforcement Learning
  1. [Class slides with annotations]
Project Feb 26 Project Milestone due, 11:59pm
Lecture Mar 2 Guest Lecture: Craig Boutilier
Exam Mar 4 In-class Quiz
Lecture Mar 9 Monte Carlo Tree Search
  • Draft Slides , Class slides with annotations]
    		•
    Project Mar 11 Poster presentations Class attendance required, requested to stay for 3 hrs.
    11:30am-2:30pm in the Huang Foyer (area outside of NVIDIA auditorium).
    Project Mar 18 Project final paper due, 11:59pm

    Lecture Notes

    This section contains the CS234 course notes created during the Winter 2018 and 2019 offerings of the course. You are welcome to use these to support your learning but they are not part of the official material of the class, and there may be unintended typos or errors. The lecture slides and videos are the primary course material and those (not the lecture notes) specify the material that is the focus of the current class.

    Git repositories for lecture notes

    There are two versions of git repositories for the lecture notes, which are hosted on AFS at the following links:
    Cloning the repositories

    Type the commands shown below to clone each repository using git:
    Adding a new lecture

    To add a new lecture note please follow the instructions below:

    1. Copy the file template.tex in the repository and rename appropriately (e.g. lectureX.tex).
    2. Add packages when needed in the preamble section.
    3. Fill out the information in the config section of the file.
    4. Type out the lecture content.
    5. If you are making notes for lecture X, put any images needed in the directory images/lectureX.
    6. Possibly add frequently needed packages to template.tex.

    Student contributions

    Students can contribute to the editable repository, which will be monitored by the course staff to assure that the updates are correct, and when approved they will be copied into the stable repository. We welcome anyone to correct any typos, add additional sections which they feel may be missing, add figures or any other additions that you think will improve the lecture notes.