Course Description

This course teaches advanced principles for endowing mobile autonomous robots with capabilities to autonomously learn new skills and to physically interact with the environment and with humans. Concepts that will be covered in the course are: Reinforcement Learning (RL) and its relationship to optimal control, contact and dynamics models for prehensile and non-prehensile robot manipulation, as well as imitation learning and human intent inference. Students will learn the theoretical foundations for these concepts. Prerequisites: CS106A or equivalent, CME 100 or equivalent (for linear algebra), CME 106 or equivalent (for probability theory), and AA 174A/274A.

Instructors

Prof. Jeannette Bohg	Prof. Marco Pavone	Prof. Dorsa Sadigh

Course Assistants

Annie Xie	Yilun Hao

Meeting Times

Lectures meet on Mondays and Wednesdays from 1:30pm to 3:00pm at Thornton 102.

Prof. Bohg's office hours are on Wednesdays 3:00pm to 4:00pm in Gates 244 and on Zoom.
Prof. Pavone's office hours are on Fridays 2:00pm to 3:00pm by appointment.
Prof. Sadigh's office hours are on Fridays 9:00am to 10:00am by appointment.
CA office hours are on

• Mondays from 5:00pm to 6:00pm (on Zoom)
• Tuesdays from 10:30am to 12:00pm (in-person at 240-101 and on Zoom)
• Fridays from 3:30pm to 5:00pm (in-person at Durand 023 and on Zoom)

Syllabus

The class syllabus can be found here.

Schedule

Subject to change. Lecture recordings will be posted on Canvas.

Blue: learning-based control and perception
Red: interaction with the physical environment
Green: interaction with humans

Week	Topic	Lecture Slides	Lecture Notes
1	(Jan 09) Course overview, intro to ML for robotics (Jan 11) Neural networks and Tensorflow tutorial (Jan 13) HW1 out	Lecture 1 Lecture 2	Lecture 1 Colab notebook
2	(Jan 16) Martin Luther King Jr. Day (no class) (Jan 18) Markov decision processes	Lecture 3	Lecture 3
3	(Jan 23) Intro to RL (Jan 25) Model-based and model-free RL for robot control	Lecture 4 Lecture 5	Lectures 4 & 5
4	(Jan 30) Learning-based perception (Feb 01) Fundamentals of grasping and manipulation I (Feb 03) HW1 due, HW2 out	Lecture 6 Lecture 7	Lecture 6 Lectures 7 & 8 & 9
5	(Feb 06) Fundamentals of grasping and manipulation II (Feb 08) Learning-based grasping and manipulation (Feb 10) Exam 1	Lecture 8 Lecture 9	Optional Readings: - Data-Driven Grasp Synthesis - Learning Hand-Eye Coordination for Robotic Grasping - Dex-Net 2.0 - Learning to Grasp Novel Objects using Vision
6	(Feb 13) Interactive Perception (Feb 15) Guest Lecture by Karol Hausman (Google Brain, Stanford) Karol's website (Feb 17) HW2 due, HW3 out	Lecture 10	Optional Readings: - Interactive Perception - Making Sense of Vision and Touch - Combining Learned and Analytical Models - Motion-Based Object Segmentation
7	(Feb 20) Presidents' Day (no class) (Feb 22) Imitation learning I (Feb 24) Exam 2	Lecture 12	Lectures 12 & 13 & 14 & 15
8	(Feb 27) Imitation learning II (Mar 01) Learning from human feedback	Lecture 13 Lecture 14
9	(Mar 06) Interaction-aware learning, planning, and control (Mar 08) Guest lecture by Sidd Karamcheti (Mar 10) HW3 due	Lecture 15
10	(Mar 13) Shared autonomy (Mar 15) Paper presentations (Mar 17) Exam 3	Lecture 17