Exploration in Reinforcement Learning Reading List

Curated by Mouhssine Rifaki | Stanford Electrical Engineering | Last updated August 2026

How agents learn to explore efficiently in large, sparse-reward environments.

Exploration in Reinforcement Learning: 10 key papers

1. Unifying Count-Based Exploration and Intrinsic Motivation
   Bellemare et al. arXiv 2016.
2. Never Give Up: Learning Directed Exploration Strategies
   Badia et al. arXiv 2020.
3. First return, then explore
   Ecoffet et al. arXiv 2020.
4. Exploration by Random Network Distillation
   Burda et al. arXiv 2018.
5. Deep Exploration via Bootstrapped DQN
   Osband et al. arXiv 2016.
6. Noisy Networks for Exploration
   Fortunato et al. arXiv 2017.
7. Parameter Space Noise for Exploration
   Plappert et al. arXiv 2017.
8. Go-Explore: a New Approach for Hard-Exploration Problems
   Ecoffet et al. arXiv 2019.
9. Is Q-learning Provably Efficient?
   Jin et al. arXiv 2018.
10. Provably Efficient Maximum Entropy Exploration
    Hazan et al. arXiv 2018.

Related reading lists

• Curiosity-Driven Exploration
• Intrinsic Motivation & Empowerment
• Linear MDPs & Linear Function Approximation
• Bayesian RL & Thompson Sampling
• Block MDPs & Latent State Discovery
• Offline Reinforcement Learning
• In-Context RL & Prompting
• Few-Shot RL & Meta-Learning