Distributional Reinforcement Learning Reading List

Curated by Mouhssine Rifaki | Stanford Electrical Engineering | Last updated August 2026

Instead of collapsing uncertainty into a single expectation, distributional RL learns the full return distribution. These papers show how categorical, quantile, and implicit representations improve stability, exploration, and risk sensitivity.

Distributional Reinforcement Learning: 10 key papers

1. A Distributional Perspective on Reinforcement Learning
   Bellemare et al. arXiv 2017.
2. Distributional Reinforcement Learning with Quantile Regression
   Dabney et al. arXiv 2017.
3. Implicit Quantile Networks for Distributional Reinforcement Learning
   Dabney et al. arXiv 2018.
4. Fully Parameterized Quantile Function for Distributional Reinforcement Learning
   Yang et al. arXiv 2019.
5. An Analysis of Categorical Distributional Reinforcement Learning
   Rowland et al. arXiv 2018.
6. A Comparative Analysis of Expected and Distributional Reinforcement Learning
   Lyle et al. arXiv 2019.
7. Statistics and Samples in Distributional Reinforcement Learning
   Rowland et al. arXiv 2019.
8. Distributional Reinforcement Learning for Efficient Exploration
   Mavrin et al. arXiv 2019.
9. Distributed Distributional Deterministic Policy Gradients
   Barth-Maron et al. arXiv 2018.
10. Distributional Reinforcement Learning via Moment Matching
    Nguyen et al. arXiv 2020.

Related reading lists

• Sample Complexity in Reinforcement Learning
• Neuroscience-Inspired Reinforcement Learning
• Exploration in Reinforcement Learning
• Domain Adaptation & Domain Generalization
• Bayesian Deep Learning
• Multi-Armed Bandits & Best-Arm Identification
• Uncertainty Quantification & Conformal Prediction
• Imitation Learning and Inverse RL