Yuke Zhu

I am a PhD student at Stanford University. My research objective is to teach robots to understand and to interact with the visual world.

My research resides at the intersection of computer vision, machine learning, and robotics, with a focus on visual knowledge and deep reinforcement learning. I work in Stanford Vision Lab with Prof. Fei-Fei Li and Prof. Silvio Savarese. Prior to coming to Stanford, I received a BSc. degree from Simon Fraser University and a BEng. degree from Zhejiang University.

Email: yukez@cs.stanford.edu

Gates Computer Science Building, Room 242
353 Serra Mall, Stanford University
Stanford, CA 94305-9025, USA


[new] We have released our new paper on one-shot visual imitation with neural task graphs.

We have one workshop and two papers accepted in RSS 2018.

We released our work (with DeepMind) on end-to-end visuomotor learning for robot manipulation.

Our work on Neural Task Programming is accepted in ICRA 2018. Check out our video here.

We publicly released the AI2-THOR, an open-source platform for Visual AI.

Selected Publications

  • Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration
    De-An Huang*, Suraj Nair*, Danfei Xu*, Yuke Zhu, Animesh Garg, Li Fei-Fei, Silvio Savarese,
    Juan Carlos Niebles
  • Reinforcement and Imitation Learning for Diverse Visuomotor Skills
    Yuke Zhu, Ziyu Wang, Josh Merel, Andrei Rusu, Tom Erez, Serkan Cabi, Saran Tunyasuvunakool,
    János Kramár, Raia Hadsell, Nando de Freitas, Nicolas Heess
    RSS 2018
  • Learning Task-Oriented Grasping for Tool Manipulation with Simulated Self-Supervision
    Kuan Fang, Yuke Zhu, Animesh Garg, Virja Mehta, Andrey Kuryenkov, Li Fei-Fei, Silvio Savarese
    RSS 2018
  • Neural Task Programming: Learning to Generalize Across Hierarchical Tasks
    Danfei Xu*, Suraj Nair*, Yuke Zhu, Julian Gao, Animesh Garg, Li Fei-Fei, Silvio Savarese
    ICRA 2018
  • ADAPT: Zero-Shot Adaptive Policy Transfer for Stochastic Dynamical Systems
    James Harrison*, Animesh Garg*, Boris Ivanovic, Yuke Zhu, Silvio Savarese, Li Fei-Fei, Marco Pavone
    ISRR 2017
  • Visual Semantic Planning using Deep Successor Representations
    Yuke Zhu*, Daniel Gordon*, Eric Kolve, Dieter Fox, Li Fei-Fei, Abhinav Gupta, Roozbeh Mottaghi, Ali Farhadi
    ICCV 2017
  • Adversarially Robust Policy Learning: Active Construction of Physically-Plausible Perturbations
    Ajay Mandlekar*, Yuke Zhu*, Animesh Garg*, Li Fei-Fei, Silvio Savarese
    IROS 2017
  • Scene Graph Generation by Iterative Message Passing
    Danfei Xu, Yuke Zhu, Christopher B. Choy, Li Fei-Fei
    CVPR 2017
  • Knowledge Acquisition for Visual Question Answering via Iterative Querying
    Yuke Zhu, Joseph J. Lim, Li Fei-Fei
    CVPR 2017
  • Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning
    Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph J. Lim, Abhinav Gupta, Li Fei-Fei, Ali Farhadi
    ICRA 2017
  • Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen,
    Yannis Kalanditis, Li-Jia Li, David A. Shamma, Michael Bernstein, Li Fei-Fei
    IJCV 2017
  • Visual7W: Grounded Question Answering in Images
    Yuke Zhu, Oliver Groth, Michael Bernstein, Li Fei-Fei
    CVPR 2016
  • Action Recognition by Hierarchical Mid-level Action Elements
    Tian Lan*, Yuke Zhu*, Amir Roshan Zamir, Silvio Savarese [* indicates equal contribution]
    ICCV 2015
  • Building a Large-scale Multimodal Knowledge Base System for Answering Visual Queries
    Yuke Zhu, Ce Zhang, Christopher Ré, Li Fei-Fei
  • Reasoning About Object Affordances in a Knowledge Base Representation
    Yuke Zhu, Alireza Fathi, Li Fei-Fei
    ECCV 2014

Teaching Experience

  • Teaching Assistant

    Spring 2013-2014 | Stanford, CA, USA

    CS 431: High-Level Vision: Behaviors, Neurons and Computational Models

    Summer 2013-2014 | Stanford, CA, USA

    CS 193C: Client-Side Internet Technologies

    Fall 2014-2015 | Stanford, CA, USA

    CS 131: Computer Vision: Foundations and Applications

    Winter 2014-2015 | Stanford, CA, USA

    CS 231N: Convolutional Neural Networks for Visual Recognition

Working Experience

  • Research Intern

    Jun - Sept 2017 | London, England, United Kingdom

  • Research Intern

    Jun - Sept 2016 | Seattle, WA, USA

    Allen Institute for Artificial Intelligence
  • Research Intern

    May - Aug 2015 | Venice, CA, USA

    Snap Inc.
  • Software Engineer Intern

    Apr - Jul 2013 | San Francisco, CA, USA

    Twitter Inc.
  • Research Assistant

    Dec 2011 - Apr 2013 | Vancouver, BC, Canada

    SFU Computational Logic Lab

    Jan 2012 - Apr 2013 | Vancouver, BC, Canada

    SFU Vision and Media Lab