Robust & Risk-Sensitive RL Reading List

Curated by Mouhssine Rifaki | Stanford Electrical Engineering | Last updated August 2026

Robust and risk-sensitive reinforcement learning asks what happens when the world does not match the model. This list brings together adversarial formulations, robust MDP theory, constrained optimization, and risk measures that turn fragile policies into dependable ones.

Robust & Risk-Sensitive RL: 10 key papers

1. Robust Markov Decision Processes
   Wiesemann et al. Mathematics of Operations Research 2013.
2. Wasserstein Robust Reinforcement Learning
   Abdullah et al. arXiv 2019.
3. Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations
   Zhang et al. arXiv 2020.
4. Action Robust Reinforcement Learning and Applications in Continuous Control
   Tessler et al. arXiv 2019.
5. Sample Complexity of Robust Reinforcement Learning with a Generative Model
   Panaganti and Kalathil. arXiv 2021.
6. Online Robust Reinforcement Learning with Model Uncertainty
   Wang and Zou. arXiv 2021.
7. Policy Gradient Method For Robust Reinforcement Learning
   Wang and Zou. arXiv 2022.
8. Robust Constrained Reinforcement Learning
   Wang et al. arXiv 2022.
9. Distributionally Robust Deep Q-Learning
   Lu et al. arXiv 2025.
10. Robust Reinforcement Learning using Offline Data
    Panaganti et al. arXiv 2022.

Related reading lists

• Adversarial Reinforcement Learning & Robustness to Adversaries
• Safe Reinforcement Learning & Constrained MDPs
• Linear MDPs & Linear Function Approximation
• Optimal Transport for Machine Learning
• Optimal Control & LQR/LQG
• Off-Policy Evaluation & Offline Evaluation
• Offline Imitation Learning
• Transformers for Sequential Decision Making