Multi-Agent Reinforcement Learning Reading List

Curated by Mouhssine Rifaki | Stanford Electrical Engineering | Last updated April 2026

Core papers for understanding multi-agent reinforcement learning, from foundational algorithms to modern scalable methods.

1. Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
   Lowe et al.. NeurIPS 2017.
2. Counterfactual Multi-Agent Policy Gradients
   Foerster et al.. AAAI 2018.
3. QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
   Rashid et al.. ICML 2018.
4. The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games
   Yu et al.. NeurIPS 2022.
5. Value-Decomposition Networks for Cooperative Multi-Agent Learning
   Sunehag et al.. AAMAS 2018.
6. OpenSpiel: A Framework for Reinforcement Learning in Games
   Lanctot et al.. arXiv 2019.
7. PettingZoo: Gym for Multi-Agent Reinforcement Learning
   Terry et al.. NeurIPS 2021.
8. Grandmaster Level in StarCraft II Using Multi-Agent Reinforcement Learning
   Vinyals et al.. Nature 2019.
9. Emergent Complexity via Multi-Agent Competition
   Bansal et al.. ICLR 2018.
10. Mean Field Multi-Agent Reinforcement Learning
    Yang et al.. ICML 2018.
11. Scalable Multi-Agent Reinforcement Learning through Intelligent Information Aggregation
    Nayak et al.. ICML 2023.
12. Learning to Communicate with Deep Multi-Agent Reinforcement Learning
    Foerster et al.. NeurIPS 2016.