Yuke Zhu
 

I am a PhD student at Stanford University. My research objective is to build general-purpose robots that understand and interact with the visual world.

My research lies at the intersection of computer vision, machine learning, and robotics, with a focus on robotics perception and deep reinforcement learning. I work in Stanford Vision and Learning Lab with Prof. Fei-Fei Li and Prof. Silvio Savarese. I am also a member of Stanford People, AI & Robots Group. Prior to coming to Stanford, I received a BSc. degree from Simon Fraser University and a BEng. degree from Zhejiang University.

Email: yukez@cs.stanford.edu

Gates Computer Science Building, Room 242
353 Serra Mall, Stanford University
Stanford, CA 94305-9025, USA


News

[new] We released our code and data for Surreal and RoboTurk, which are covered by a Stanford News article.

[new] We released our new paper on multimodal learning for robot manipulation.

We have two papers on robotic manipulation accepted in CoRL 2018.

We have released our new paper on one-shot visual imitation with neural task graphs.

One workshop and two papers are accepted in RSS 2018.


Selected Publications

  • Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for
    Contact-Rich Tasks
    Michelle A. Lee*, Yuke Zhu*, Krishnan Srinivasan, Parth Shah, Silvio Savarese, Li Fei-Fei, Animesh Garg,
    Jeannette Bohg
    arXiv:1810.10191
  • SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark
    Linxi Fan*, Yuke Zhu*, Jiren Zhu, Zihua Liu, Orien Zeng, Anchit Gupta, Joan Creus-Costa, Silvio Savarese,
    Li Fei-Fei
    CoRL 2018
  • RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation
    Ajay Mandlekar, Yuke Zhu, Animesh Garg, Jonathan Booher, Max Spero, Albert Tung, Julian Gao,
    John Emmons, Anchit Gupta, Emre Orbay, Silvio Savarese, Li Fei-Fei
    CoRL 2018
  • Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration
    De-An Huang*, Suraj Nair*, Danfei Xu*, Yuke Zhu, Animesh Garg, Li Fei-Fei, Silvio Savarese,
    Juan Carlos Niebles
    arXiv:1807.03480
  • Reinforcement and Imitation Learning for Diverse Visuomotor Skills
    Yuke Zhu, Ziyu Wang, Josh Merel, Andrei Rusu, Tom Erez, Serkan Cabi, Saran Tunyasuvunakool,
    János Kramár, Raia Hadsell, Nando de Freitas, Nicolas Heess
    RSS 2018
  • Learning Task-Oriented Grasping for Tool Manipulation with Simulated Self-Supervision
    Kuan Fang, Yuke Zhu, Animesh Garg, Virja Mehta, Andrey Kuryenkov, Li Fei-Fei, Silvio Savarese
    RSS 2018
  • Neural Task Programming: Learning to Generalize Across Hierarchical Tasks
    Danfei Xu*, Suraj Nair*, Yuke Zhu, Julian Gao, Animesh Garg, Li Fei-Fei, Silvio Savarese
    ICRA 2018
  • ADAPT: Zero-Shot Adaptive Policy Transfer for Stochastic Dynamical Systems
    James Harrison*, Animesh Garg*, Boris Ivanovic, Yuke Zhu, Silvio Savarese, Li Fei-Fei, Marco Pavone
    ISRR 2017
  • Visual Semantic Planning using Deep Successor Representations
    Yuke Zhu*, Daniel Gordon*, Eric Kolve, Dieter Fox, Li Fei-Fei, Abhinav Gupta, Roozbeh Mottaghi, Ali Farhadi
    ICCV 2017
  • Scene Graph Generation by Iterative Message Passing
    Danfei Xu, Yuke Zhu, Christopher B. Choy, Li Fei-Fei
    CVPR 2017
  • Knowledge Acquisition for Visual Question Answering via Iterative Querying
    Yuke Zhu, Joseph J. Lim, Li Fei-Fei
    CVPR 2017
  • Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning
    Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph J. Lim, Abhinav Gupta, Li Fei-Fei, Ali Farhadi
    ICRA 2017
  • Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen,
    Yannis Kalanditis, Li-Jia Li, David A. Shamma, Michael Bernstein, Li Fei-Fei
    IJCV 2017
  • Visual7W: Grounded Question Answering in Images
    Yuke Zhu, Oliver Groth, Michael Bernstein, Li Fei-Fei
    CVPR 2016
  • Reasoning About Object Affordances in a Knowledge Base Representation
    Yuke Zhu, Alireza Fathi, Li Fei-Fei
    ECCV 2014

Teaching Experience

  • Teaching Assistant

    Spring 2013-2014 | Stanford, CA, USA

    CS 431: High-Level Vision: Behaviors, Neurons and Computational Models

    Summer 2013-2014 | Stanford, CA, USA

    CS 193C: Client-Side Internet Technologies

    Fall 2014-2015 | Stanford, CA, USA

    CS 131: Computer Vision: Foundations and Applications

    Winter 2014-2015 | Stanford, CA, USA

    CS 231N: Convolutional Neural Networks for Visual Recognition

Working Experience

  • Research Intern

    Jun - Sept 2017 | London, England, United Kingdom

    Deepmind
  • Research Intern

    Jun - Sept 2016 | Seattle, WA, USA

    Allen Institute for Artificial Intelligence
  • Research Intern

    May - Aug 2015 | Venice, CA, USA

    Snap Inc.
  • Software Engineer Intern

    Apr - Jul 2013 | San Francisco, CA, USA

    Twitter Inc.
  • Research Assistant

    Dec 2011 - Apr 2013 | Vancouver, BC, Canada

    SFU Computational Logic Lab

    Jan 2012 - Apr 2013 | Vancouver, BC, Canada

    SFU Vision and Media Lab