Meta-Reinforcement Learning Reading List

Curated by Mouhssine Rifaki | Stanford Electrical Engineering | Last updated August 2026

Meta-RL asks an agent to learn how to learn—adapting to new tasks from few episodes. This list spans gradient-based adaptation, probabilistic context inference, and black-box memory approaches that define the field.

Meta-Reinforcement Learning: 10 key papers

1. RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
   Duan et al. arXiv 2016.
2. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
   Finn et al. arXiv 2017.
3. Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables
   Rakelly et al. arXiv 2019.
4. ProMP: Proximal Meta-Policy Search
   Rothfuss et al. arXiv 2018.
5. VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning
   Zintgraf et al. arXiv 2019.
6. Improving Generalization in Meta Reinforcement Learning using Learned Objectives
   Kirsch et al. arXiv 2019.
7. Discovering Reinforcement Learning Algorithms
   Oh et al. arXiv 2020.
8. Meta-Gradient Reinforcement Learning
   Xu et al. arXiv 2018.
9. Meta-Q-Learning
   Fakoor et al. arXiv 2019.
10. Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning
    Nagabandi et al. arXiv 2018.

Related reading lists

• Few-Shot RL & Meta-Learning
• Long-Context & Memory Transformers
• Causal Inference & RL
• Memory-Augmented RL & Recurrent Policies
• Domain Adaptation & Domain Generalization
• Zero-Shot RL & Generalization
• Neuroscience-Inspired Reinforcement Learning
• Monte Carlo Tree Search