Research

Research in SISL spans research topics, both fundamental and applied, that examine a wide range of challenges in engineering intelligent systems. This page summarizes a few major research themes.

Autonomous Driving

One of our major focus areas is autonomous driving. We are interested in issues related to modeling, perception, and planning.

Modeling

Autonomous vehicles must be able to predict the behavior of human driven vehicles to maintain both safety and efficiency. We derive human driver models from recorded vehicle trajectories. Our research has focused on the underlying theory of learning from demonstrations; models for different modalities of human driving behavior including style and intent; learning representations and methodologies; and validation of these models.

Planning

Autonomous vehicles require efficient algorithms that reason about the potential locations of pedestrians and other vehicles along with their motion over time. To address issues with computational tractability, we have explored different decomposition methods that provide decision strategies that are close to optimal.

Perception

An accurate perception system is necessary for autonomous vehicles to navigate in cluttered dynamic environments. We use temporal deep learning models to predict the future environment state given a history of observed LiDAR measurements. Experiments on a comprehensive dataset indicate that our methods can build effective representations of the environment evolution with time. We also address the problem of object occlusion by treating observable human-object interactions as additional sensor information. We have developed a multi-sensor framework based on an evidential theory for human drivers visible to the autonomous vehicle to infer the occupancy state within occluded regions.

Selected References:

T. A. Wheeler, P. Robbel, and M. J. Kochenderfer, “A probabilistic framework for microscopic traffic propagation,” in IEEE International Conference on Intelligent Transportation Systems (ITSC), Las Palmas de Gran Canaria, Spain, 2015.
A. Kuefler, J. Morton, T. A. Wheeler, and M. J. Kochenderfer, “Imitating driver behavior with generative adversarial networks,” in IEEE Intelligent Vehicles Symposium (IV), 2017.
D. Phillips, T. A. Wheeler, and M. J. Kochenderfer, “Generalizable intention prediction of human drivers at intersections,” in IEEE Intelligent Vehicles Symposium (IV), 2017.
J. Morton, T. A. Wheeler, and M. J. Kochenderfer, “Analysis of recurrent neural networks for probabilistic modeling of driver behavior,” IEEE Transactions on Intelligent Transportation Systems, vol. 18, iss. 5, pp. 1289-1298, 2017.
M. Bouton, A. Cosgun, and M. J. Kochenderfer, “Belief state planning for autonomously navigating urban intersections,” in IEEE Intelligent Vehicles Symposium (IV), 2017.
M. Bouton, K. Julian, A. Nakhaei, K. Fujimura, and M. J. Kochenderfer, “Utility decomposition with deep corrections for scalable planning under uncertainty,” in International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2018.
R. Bhattacharyya, D. P. Phillips, B. Wulfe, J. Morton, A. Kuefler, and M. J. Kochenderfer, “Multi-agent imitation learning for driving simulation,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018.
M. Kelly, C. Sidrane, K. Driggs-Campbell, and M. J. Kochenderfer, “Safe interactive imitation learning from humans,” in Imitation Learning and its Challenges in Robotics Workshop, Advances in Neural Information Processing Systems (NIPS), 2018.

POMDPs

An important mathematical framework for sequential decision making under uncertainty is the Partially Observable Markov Decision Process (POMDP). It allows the representation of environments where there is uncertainty about both the current state of the environment and the outcomes of decisions made by the agent. We developed POMDPs.jl, a comprehensive Julia interface for representing, defining, and solving discrete and continuous POMDPs. We have also produced POMCPOW, a state-of-the-art method for solving large or continuous-space POMDPs in an online manner.

Selected References:

M. Egorov, Z. N. Sunberg, E. Balaban, T. A. Wheeler, J. K. Gupta, and M. J. Kochenderfer, “POMDPs.jl: A framework for sequential decision making under uncertainty,” Journal of Machine Learning Research, vol. 18, iss. 26, p. 1–5, 2017.
Z. N. Sunberg and M. J. Kochenderfer, “Online algorithms for POMDPs with continuous state, action, and observation spaces,” in International Conference on Automated Planning and Scheduling (ICAPS), 2018.

Deep Reinforcement Learning

The powerful representational properties of deep networks have enabled reinforcement learning to scale to previously intractable high-dimensional problems. We have built upon recent developments in the field to scale learning in cooperative multiagent systems for complex high-dimensional tasks. We are also working on using the hierarchical structure of complex tasks to reduce the sample complexity of such methods.

Selected References:

J. K. Gupta, M. Egorov, and M. J. Kochenderfer, “Cooperative multi-agent control using deep reinforcement learning,” in Adaptive Learning Agents Workshop, International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2017.
B. Wu, J. Gupta, and M. J. Kochenderfer, “Model primitive hierarchical lifelong reinforcement learning,” in International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2019.
K. D. Julian and M. J. Kochenderfer, “Autonomous distributed wildfire surveillance using deep reinforcement learning,” in AIAA Guidance, Navigation, and Control Conference (GNC), 2018.

Safety and Validation

Safety critical systems require safe and trusted algorithms. As part of the Center for AI Safety, we are working to design safe decision strategies and to develop algorithms to efficiently validate them.

Neural Network Verification

Controllers based on Deep Neural Networks (DNNs) have the potential to outperform traditional software-based approaches for tasks ranging from automated language translation to self-driving cars. However, the lack of interpretability of DNNs poses a significant challenge to their adoption for safety-critical applications such as aircraft collision avoidance systems. To address this problem, we have collaborated with other researchers to develop a tool (Reluplex) that mathematically verifies properties of the DNN. We are continuing to develop new use cases and applications for this tool.

Adaptive Stress Testing

Adaptive stress testing (AST) is an approach for validating safety-critical autonomous control systems such as self-driving cars and unmanned aircraft. The approach involves searching for the most likely failure scenarios according to some measure of the likelihood of occurrence. To enhance efficiency when searching the enormous space of possible scenarios, AST applies reinforcement learning to a black-box simulation of the autonomous agent and its environment.

Selected References:

K. D. Julian, J. Lopez, J. S. Brush, M. P. Owen, M. J. Kochenderfer, “Policy compression for aircraft collision avoidance systems,”in Digital Avionics Systems Conference (DASC), 2016.
G. Katz, C. Barrett, D. L. Dill, K. D. Julian, and M. J. Kochenderfer, “Reluplex: An efficient SMT solver for verifying deep neural networks,” in International Conference on Computer-Aided Verification, 2017.
G. Katz, C. Barrett, D. L. Dill, K. D. Julian, and M. J. Kochenderfer, “Towards proving the adversarial robustness of deep neural networks,” in Formal Verification of Autonomous Vehicles Workshop, 2017.
L. Kuper, G. Katz, J. Gottschlich, K. Julian, C. Barrett, and M. J. Kochenderfer, “Toward scalable verification for safety-critical deep networks,” in SysML, 2018.
R. Lee, M. J. Kochenderfer, O. J. Mengshoel, G. P. Brat, and M. P. Owen, “Adaptive Stress Testing of Airborne Collision Avoidance Systems,” in Digital Avionics Systems Conference (DASC), 2015.
M. Koren, S. Alsaif, R.Lee, and M. J. Kochenderfer, “Adaptive Stress Testing for Autonomous Vehicles,” IEEE Intelligent Vehicles Symposium (IV), 2018