AA 174B / AA 274B / CS 237B / EE 260B

Principles of Robot Autonomy II

Winter 2020

Course Description

This course teaches advanced principles for endowing mobile autonomous robots with capabilities to autonomously learn new skills and to physically interact with the environment and with humans. It also provides an overview of different robot system architectures. Concepts that will be covered in the course are: Reinforcement Learning and its relationship to optimal control, contact and dynamics models for prehensile and non-prehensile robot manipulation, imitation learning and human intent inference, as well as different system architectures and their verification. Students will earn the theoretical foundations for these concepts and implementnthem on mobile manipulation platforms. In homeworks, the Robot Operating System (ROS) will be used extensively for demonstrations and hands-on activities. Prerequisites: CS106A or equivalent, CME 100 or equivalent (for linear algebra), CME 106 or equivalent (for probability theory), and AA 174A/274A.

Course Assistants

Ashar Alam Erdem Bıyık Jenna Lee Toki Migimatsu

Meeting Times

Lectures meet on Mondays and Wednesdays from 1:30pm to 2:50pm in Packard 101.

Sections are on Mondays 10:30am to 12:30pm, Tuesdays from 12:00pm to 2:00pm, Thursdays from 5:00pm to 7:00pm in the Skilling Lab.

Prof. Bohg's office hours are on Fridays 1:00pm to 2:00pm, after class, and by appointment, in Gates 140.
Prof. Pavone's office hours are on Tuesdays 1:00pm to 2:00pm, after class, and by appointment, in Durand 261.
Prof. Sadigh's office hours are on Fridays 9:00am to 10:00am, after class, and by appointment, in Gates 142.
CA office hours are on Tuesdays from 10:00am to 12:00pm and Fridays from 3:00pm to 5:00pm in Durand 023; and on Mondays from 09:00am to 10:00am online for SCPD students.


The class syllabus can be found here.

If you are a Stanford Center for Professional Development (SCPD) student, please see this syllabus instead, namely the parts about sections and the final project.


Subject to change. Lecture videos will be posted by the SCPD.

Week Topic Lecture Slides Sections
1 (Jan 06) Course overview, intro to ML for robotics
(Jan 08) Markov decision processes, intro to RL
HW1 out
Lecture 1
Lecture 2
2 (Jan 13) Model-based and model-free RL for robot control
(Jan 15) Learning-based perception
Lecture 3
Lecture 4
Section 1 Slides
Section 1 Handout
3 (Jan 20) Martin Luther King Jr. Day (no class)
(Jan 22) System architectures
HW1 due, HW2 out
Lecture 5
- Reading: Chapter 12 of this book
- Case Study
- Reading: This paper
4 (Jan 27) Specifications and model checking
(Jan 29) Formal verification of neural networks (by Clark Barrett)
Lecture 6
- Reading: This paper
Lecture 7
- Model Checking
- Verification of Neural Networks
5 (Feb 03) System-level verification via stress testing (by Thomas Kühbeck)
(Feb 05) Fundamentals of grasping
HW2 due, HW3 out
Lecture 8
Lecture 9
- Reading: Chapter 38 of this book
Section 2 Slides
Section 2 Handout
6 (Feb 10) Grasp force optimization and planar pushing
(Feb 12) Learning-based grasping and manipulation
Lecture 10
- Reading: Chapter 37 of this book
- Reading: This paper
- Reading: This paper
- Reading: This paper
Lecture 11
- Reading: This paper
- Reading: This paper
- Reading: This paper
- Reading: This paper
7 (Feb 17) Presidents' Day (no class)
(Feb 19) Interactive perception
Final project released
Lecture 12
- Reading: This paper
- Reading: This paper
- Reading: This paper
Section 3 Slides
Section 3 Handout
8 (Feb 24) Imitation learning I
(Feb 26) Imitation learning II
HW3 due, HW4 out
Lecture 13
- ALVINN (Behavior Cloning), 1989
- Algorithms for IRL, 2000
- Apprenticeship Learning, 2004
- Maximum Margin Planning, 2006
- DAgger, 2011
- CoIL, 2018
Lecture 14
- MaxEnt IRL, 2010
- This Paper
- This Paper
- This Paper
9 (Mar 02) Learning from pairwise comparisons and physical feedback
(Mar 04) Interaction-aware control, intent inference, and shared autonomy
Friday: Final project check-in
Lecture 15
- Reading: This paper
Lecture 16
- Reading: This Paper
- Reading: This Paper
- Reading: This Paper
10 (Mar 09) Final exam (in class)
(Mar 11) Final Project Demo (no class)
HW4 due