Multi-Agent Reinforcement Learning Reading List

Curated by Mouhssine Rifaki | Stanford Electrical Engineering | Last updated August 2026

Core papers for understanding multi-agent reinforcement learning, from foundational algorithms to modern scalable methods.

Multi-Agent Reinforcement Learning: 10 key papers

1. Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
   Lowe et al. arXiv 2017.
2. QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
   Rashid et al. arXiv 2018.
3. Counterfactual Multi-Agent Policy Gradients
   Foerster et al. arXiv 2017.
4. Value-Decomposition Networks For Cooperative Multi-Agent Learning
   Sunehag et al. arXiv 2017.
5. The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games
   Yu et al. arXiv 2021.
6. Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
   Foerster et al. arXiv 2017.
7. QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning
   Son et al. arXiv 2019.
8. The StarCraft Multi-Agent Challenge
   Samvelyan et al. arXiv 2019.
9. Multi-Agent Reinforcement Learning is a Sequence Modeling Problem
   Wen et al. arXiv 2022.
10. Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?
    de Witt et al. arXiv 2020.

Related reading lists

• Communication in Multi-Agent RL
• Game Theory for Multi-Agent Systems
• Policy Gradient Methods
• Emergent Behavior & Self-Organization
• Hindsight Experience Replay & Relabeling
• Autonomous Driving with RL
• Opponent Modeling & Theory of Mind
• Mean-Field Games & Mean-Field Control