|
Syllabus
Please find below all papers that we will discuss and review during class. Further below, please find additional supplementary background material.
Note: The course sessions will review papers that are available for free or through Stanford's subscription.
To access readings while off campus, connect to the Stanford network via VPN.
| Date | Topic | Foundational Paper | Tutorial Papers | Assignemts Due |
| Sep 24 | Introduction and Course Logistics | | | |
|
Sep 26 | Topic 1
Classic Contact-Based Grasping | Planning Optimal Grasps. C Ferrari and J Canny. IEEE International Conference on Robotics and Automation. 1992. | FRoGGeR: Fast Robust Grasp Generation via the Min-Weight Metric Albert H. Li, Preston Culbertson, Joel W. Burdick, Aaron D. Ames. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IROS 2023.
Automatic grasp planning using shape primitives. AT Miller, S Knoop, HI Christensen, PK Allen. IEEE International Conference on Robotics and Automation. 2003.
Classical Grasp Quality Evaluation: New Theory and Algorithms. Florian Pokorny and Danica Kragic. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2013. | Paper review and tutorial question (11:59pm day before) |
|
Oct 1 | Guest Lecture by Zachary Robertson: LLMs for Peer Review | | | |
|
Oct 3 | Topic 2
Grasp Synergies and other Grasp Taxonomies | Postural Hand Synergies for Tool Use. M Santello, M Flanders, J F Soechting. J Neurosci. 1998 Dec 1;18(23):10105-15. | Hand posture subspaces for dexterous robotic grasping. MT Ciocarlie, PK Allen. The International Journal of Robotics Research 28 (7), 851-867
Adaptive synergies for the design and control of the Pisa/IIT SoftHand. Catalano MG, Grioli G, Farnioli E, Serio A, Piazza C, Bicchi A. n International Journal of Robotics Research (IJRR), 2014.
The GRASP Taxonomy of Human Grasp Types. T. Feix, J. Romero, H. Schmiedmayer, A. M. Dollar and D. Kragic, IEEE Transactions on Human-Machine Systems, vol. 46, no. 1, pp. 66-77, Feb. 2016. | Paper review and tutorial question (11:59pm day before) |
|
Oct 8 | Topic 3
Exploiting Contact Constraints | Extrinsic Dexterity: In-Hand Manipulation with External Forces. Nikhil Chavan-Dafle and Alberto Rodriguez and Robert Paolini and Bowei Tang and Siddhartha Srinivasa and Michael Erdmann and Matthew T. Mason and Ivan Lundberg and Harald Staab and Thomas Fuhlbrigge, IEEE International Conference on Robotics and Automation. 2014. | Exploitation of environmental constraints in human and robotic grasping. Clemens Eppner, Raphael Deimel, José Álvarez-Ruiz, Marianne Maertens, Oliver Brock, In International Journal of Robotics Research (IJRR)
Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity. Wenxuan Zhou, David Held. Conference on Robot Learning (CoRL). 2022.
Learning to Scaffold the Development of Robotic Manipulation Skills. Shao, L., Migimatsu, T., Bohg, J. IEEE International Conference on Robotics and Automation (ICRA). 2020. | Paper review and tutorial question (11:59pm day before) |
|
Oct 10 | Topic 4
Learning to Grasp | Conference version, for review:
Robotic Grasping of Novel Objects. Ashutosh Saxena, Justin Driemeyer, Justin Kearns, Andrew Y. Ng. In Advances in Neural Information Processing Systems (NeurIPS) , 2006.
Journal version, for reference:
Robotic Grasping of Novel Objects using Vision | Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics. Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, Michael Laskey, Richard Doan, Xinyu Liu, Juan Ojea, Ken Goldberg. Robotics: Science and Systems (RSS) 2017.
QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation. Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, Sergey Levine. Proceedings of The 2nd Conference on Robot Learning (CoRL), 2018
Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes. Martin Sundermeyer, Arsalan Mousavian, Rudolph Triebel, Dieter Fox. IEEE International Conference on Robotics and Automation (ICRA). 2021.
AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domains. Hao-Shu Fang, Chenxi Wang, Hongjie Fang, Minghao Gou, Jirong Liu, Hengxu Yan, Wenhai Liu, Yichen Xie, Cewu Lu. IEEE Transactions on Robotics, vol. 39, no. 5, pp. 3929-3945, Oct. 2023. | Paper review and tutorial question (11:59pm day before) |
|
Oct 15 | Topic 5
Trajectory Optimization for Motion Planning | Conference version, for review:
Finding Locally Optimal, Collision-Free Trajectories with Sequential Convex Optimization. John Schulman, Jonathan Ho, Alex Lee, Ibrahim Awwal, Henry Bradlow and Pieter Abbeel. Robotics: Science and Systems (RSS) 2013.
Journal version, for reference:
Motion Planning with Sequential Convex Optimization and Convex Collision Checking John Schulman, Yan Duan, Jonathan Ho, Alex Lee, Ibrahim Awwal, Henry Bradlow,Jia Pan, Sachin Patil, Ken Goldberg, Pieter Abbeel. The International Journal of Robotics Research, 33(9), 1251–1270. | Motions in Microseconds via Vectorized Sampling-Based Planning. W. Thomason, Z. Kingston and L. E. Kavraki, 2024 IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan, 2024, pp. 8749-8756.
Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving. Grady Williams, Paul Drews, Brian Goldfain, James M. Rehg, Evangelos A. Theodorou. Transactions on Robotics. 2018.
Non-Euclidean Motion Planning with Graphs of Geodesically-Convex Sets. Thomas B Cohn, Mark Petersen, Max Simchowitz, Russ Tedrake. Robotics: Science and Systems (RSS) 2023. | Paper review and tutorial question (11:59pm day before) |
|
Oct 17 | Topic 6
Reinforcement Learning for Manipulation | End-to-End Training of Deep Visuomotor Policies. Sergey Levine*, Chelsea Finn*, Trevor Darrell, Pieter Abbeel. JMLR 17, 2016. | Learning dexterous in-hand manipulation. Andrychowicz, O. M. et al. he International Journal of Robotics Research, 39(1), pp. 3–20. 2020
Reaching the limit in autonomous racing: Optimal control versus reinforcement learning. Yunlong Song , Angel Romero, Matthias Müller, Vladlen Koltun, and Davide Scaramuzza. Science Robotics 2023.
Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning. Nikita Rudin, David Hoeller, Philipp Reist, Marco Hutter. Conference on Robot Learning (CoRL), 2022. | Paper review and tutorial question (11:59pm day before) |
|
Oct 22 | Topic 7
Imitation Learning | A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. AIStats 2011. | Generalization of Motor Skills by Learning from Demonstration. P Pastor, H Hoffmann, T Asfour, S Schaal. IEEE International Conference on Robotics and Automation, 763-768
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin CM Burchfiel, Shuran Song. Proceedings of Robotics: Science and Systems. 2023.
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. Tony Zhao, Vikash Kumar, Sergey Levine, Chelsea Finn. Proceedings of Robotics: Science and Systems. 2023.
MimicPlay: Long-Horizon Imitation Learning by Watching Human Play. Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu, Anima Anandkumar. Conference on Robot Learning (CoRL), 2023. | Draft Proposal
Paper review and tutorial question (11:59pm day before) |
|
Oct 24 | Topic 8
Sensorimotor Adaptation | Online movement adaptation based on previous sensor experiences. P Pastor, L Righetti, M Kalakrishnan, S Schaal IEEE/RSJ International Conference on Intelligent Robots and Systems , 365-371 | Residual policy learning. Tom Silver, Kelsey Allen, Josh Tenenbaum, Leslie Kaelbling. arXiv, 2018.
Iterative Residual Policy for Goal-Conditioned Dynamic Manipulation of Deformable Objects. Cheng Chi, Benjamin Burchfiel, Eric Cousineau, Siyuan Feng, Shuran Song. Robotics: Science and Systems (RSS) 2022.
In-Hand Object Rotation via Rapid Motor Adaptation. Qi, Haozhi and Kumar, Ashish and Calandra, Roberto and Ma, Yi and Malik, Jitendra. Conference on Robot Learning (CoRL) 2022. | Paper review and tutorial question (11:59pm day before) |
|
Oct 29 | Topic 9
High-level Reasoning and Planning | Hierarchical Task and Motion Planning in the Now. Leslie Pack Kaelbling, Tomás Lozano-Pérez. IEEE International Conference on Robotics and Automation , 2011. | Differentiable Physics and Stable Modes for Tool-Use and Manipulation Planning. M. Toussaint, K. R. Allen, K. A. Smith, and J. B. Tenenbaum. In Proc. of Robotics: Science and Systems (R:SS 2018), 2018.
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances. Ahn et al. Conference on Robot Learning (CoRL), 2022.
UniSim: Learning Interactive Real-World Simulators. Sherry Yang, Yilun Du, Kamyar Ghasemipour, Jonathan Tompson, Leslie Kaelbling, Dale Schuurmans, Pieter Abbeel. ICLR’24. | Paper review and tutorial question (11:59pm day before) |
|
Oct 31 | Topic 10
Visual Tracking & 6D Pose Estimation | DART: Dense Articulated Real-Time Tracking. T. Schmidt, R. A. Newcombe, D. Fox. Robotics: Science and Systems (RSS) 2014 | Probabilistic Articulated Real-Time Tracking for Robot Manipulation. Garcia Cifuentes, C., Issac, J., Wüthrich, M., Schaal, S., Bohg, J. IEEE Robotics and Automation Letters (RA-L), 2(2):577-584, April 2017.
BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects. Bowen Wen, Jonathan Tremblay, Valts Blukis, Stephen Tyree, Thomas Müller, Alex Evans, Dieter Fox, Jan Kautz, Stan Birchfield. CVPR 2023.
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects. Bowen Wen, Wei Yang, Jan Kautz, Stan Birchfield. CVPR 2024. | Paper review and tutorial question (11:59pm day before) |
|
Nov 5 | No Class (Democracy Day) | | | |
|
Nov 7 | Guest Lecture by Chen Wang | | | |
|
Nov 12 | Topic 11
Interactive/Active Perception | DensePhysNet: Learning Dense Physical Object Representations via Multi-step Dynamic Interactions. Zhenjia Xu, Jiajun Wu, Andy Zeng, Joshua B. Tenenbaum, Shuran Song. Robotics: Science and Systems (RSS) 2019. | The Curious Robot: Learning Visual Representations via Physical Interactions. Lerrel Pinto, Dhiraj Gandhi, Yuanfeng Han, Yong-Lae Park, Abhinav Gupta. ECCV 2016.
SAM-RL: Sensing-Aware Model-based Reinforcement Learning via Differentiable Physics-based Simulation and Rendering. Jun Lv, Yunhai Feng, Cheng Zhang, Shuang Zhao, Lin Shao* and Cewu Lu*. Robotics: Science and Systems (RSS), 2023.
Mechanical Search on Shelves using Lateral Access X-RAY. Huang Huang*, Marcus Dominguez-Kuhne*, Jeffrey Ichnowski, Vishal Satish, Michael Danielczuk, Kate Sanders, Andrew Lee, Anelia Angelova, Vincent Vanhoucke, Ken Goldberg. IEEE International Conference on Robotics and Automation. 2021. | Paper review and tutorial question (11:59pm day before) |
|
Nov 14 | Topic 12
Tactile Sensing | GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force. Yuan, W.; Dong, S.; Adelson, E.H. Sensors 2017, 17, 2762. | Active Clothing Material Perception using Tactile Sensing and Deep Learning Wenzhen Yuan, Yuchen Mo, Shaoxiong Wang, and Edward H. Adelson. IEEE International Conference on Robotics and Automation. 2018
More Than a Feeling: Learning to Grasp and Regrasp Using Vision and Touch. Roberto Calandra, Andrew Owens, Dinesh Jayaraman, Justin Lin, Wenzhen Yuan, Jitendra Malik, Edward H. Adelson, Sergey Levine. IEEE Robotics and Automation Letters, vol. 3, no. 4, pp. 3300-3307, Oct. 2018.
Binding Touch to Everything: Learning Unified Multimodal Tactile Representations. Fengyu Yang, Chao Feng, Ziyang Chen, Hyoungseob Park, Daniel Wang, Yiming Dou, Ziyao Zeng, Xien Chen, Rit Gangopadhyay, Andrew Owens, Alex Wong. CVPR 2024. | Paper review and tutorial question (11:59pm day before) |
|
Nov 19 | Topic 13
Representation Learning | Dense object nets: Learning dense visual object descriptors by and for robotic manipulation. Peter R. Florence, Lucas Manuelli, and Russ Tedrake. In Conference on Robot Learning (CoRL), October 2018. | Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks. Michelle A. Lee, Yuke Zhu, Krishnan Srinivasan, Parth Shah, Silvio Savarese, Li Fei-Fei, Animesh Garg, Jeannette Bohg. IEEE International Conference on Robotics and Automation (ICRA) 2018.
Language Embedded Radiance Fields for Zero-Shot Task-Oriented Grasping. Adam Rashid, Satvik Sharma, Chung Min Kim, Justin Kerr, Lawrence Chen, Angjoo Kanazawa, Ken Goldberg. CoRL 2023.
Time-Contrastive Networks: Self-Supervised Learning from Video. P. Sermanet et al. Time-Contrastive Networks: Self-Supervised Learning from Video. IEEE International Conference on Robotics and Automation (ICRA) 2018. | Paper review and tutorial question (11:59pm day before) |
|
Nov 21 | Topic 14
Robotic System Building | Conference version, for review:
Four aspects of building robotic systems: lessons from the Amazon Picking Challenge 2015. Clemens Eppner, Sebastian Höfer, Rico Jonschkowski, Roberto Martín-Martín, Arne Sieverling, Vincent Wall, Oliver Brock.Robotics: Science and Systems (R:SS 2016), 2016.
Journal version, for reference:
Four aspects of building robotic systems: lessons from the Amazon Picking Challenge 2015. | Real-time Perception meets Reactive Motion Generation. Kappler, D., Meier, F., Issac, J., Mainprice, J., Garcia Cifuentes, C., Wüthrich, M., Berenz, V., Schaal, S., Ratliff, N., Bohg, J. IEEE Robotics and Automation Letters, 3(3):1864-1871, July 2018.
FlingBot: The Unreasonable Effectiveness of Dynamic Manipulations for Cloth Unfolding. Huy Ha, Shuran Song. Robotics: Science and Systems (RSS), 2023.
Deep RL at Scale: Sorting Waste in Office Buildings with a Fleet of Mobile Manipulators. Herzog et al. Robotics: Science and Systems (RSS), 2023. | Paper review and tutorial question (11:59pm day before) |
|
Nov 26 | No Class (Thanksgiving Recess) | | | |
|
Nov 28 | No Class (Thanksgiving Recess) | | | |
|
Dec 3 | Project Presentations | | | |
|
Dec 5 | Project Presentations
(final day of class) | | | |
|
Dec 6 | (no class) | | | Full Proposal (due 11:59 pm) |
|
Dec 10 | (no class) | | | Peer Reviews (due 11:59 pm)
|
Supplementary Material
Textbooks Robotics
Modern Robotics: Mechanics, Planning, and Control Kevin M. Lynch and Frank C. Park. 2017
Robotics - Modelling, Planning and Control Siciliano, B., Sciavicco, L., Villani, L., Oriolo, G. 2009
Springer Handbook of Robotics Siciliano, Bruno, Khatib, Oussama (Eds.). 2016
Contact-based grasp synthesis
Data-Driven Grasping and Learning for Manipulation
Data-Driven Grasp Synthesis - A Survey. J. Bohg, A. Morales, T. Asfour and D. Kragic. Transactions on Robotics. 2014.
Recent Advances in Robot Learning from Demonstration. Harish Ravichandar, Athanasios S. Polydoros, Sonia Chernova, and Aude Billard. Annual Review of Control, Robotics, andAutonomous Systems. 2019.
A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms. Oliver Kroemer, Scott Niekum, George Konidaris. Arxiv. 2019.
Deep Learning Approaches to Grasp Synthesis: A Review. R. Newbury et al. IEEE Transactions on Robotics. 2023.
Reinforcement Learning
Reinforcement Learning in Robotics: A Survey. J. Kober, D. Bagnell and J. Peters. International Journal of Robotics Research. 2013.
A Survey on Policy Search Algorithms for Learning Robot Controllers in a Handful of Trials. Chatzilygeroudis, K., Vassiliades, V., Stulp, F., Calinon, S. and Mouret, J.-B. IEEE Trans. on Robotics, 32:2, 328-347.
Trajectory Optimization
Dynamical Movement Primitives
Dynamical movement primitives: learning attractor models for motor behaviors. Auke Jan Ijspeert, Jun Nakanishi, Heiko Hoffmann, Peter Pastor, Stefan Schaal. Neural Compuation. 2013.
From Dynamic Movement Primitives to Associative Skill Memories. Peter Pastor, Mrinal Kalakrishnan, Franziska Meier, Freek Stulp, Jonas Buchli, Evangelos Theodorou, Stefan Schaal. Robotics and Autonomous Systems. 2013.
Visual Servoing
Interactive Perception
Foundation Models for Robotics
Recent Reviews
|