Unsupervised Environment Design Reading List

Curated by Mouhssine Rifaki | Stanford Electrical Engineering | Last updated August 2026

Automatically generating training environments to improve agent robustness and generalization.

Unsupervised Environment Design: 10 key papers

1. Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design
   Dennis et al. arXiv 2020.
2. Replay-Guided Adversarial Environment Design
   Jiang et al. arXiv 2021.
3. Evolving Curricula with Regret-Based Environment Design
   Parker-Holder et al. arXiv 2022.
4. Prioritized Level Replay
   Jiang et al. arXiv 2020.
5. Grounding Aleatoric Uncertainty for Unsupervised Environment Design
   Jiang et al. arXiv 2022.
6. Stabilizing Unsupervised Environment Design with a Learned Adversary
   Mediratta et al. arXiv 2023.
7. MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learning
   Samvelyan et al. arXiv 2023.
8. Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions
   Wang et al. arXiv 2020.
9. Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions
   Wang et al. arXiv 2019.
10. Diversity Induced Environment Design via Self-Play
    Li et al. arXiv 2023.

Related reading lists

• Monte Carlo Tree Search
• Adversarial Reinforcement Learning & Robustness to Adversaries
• LLM Agents & Tool-Augmented Language Models
• Self-Play & Population-Based Training
• Curriculum Learning & Teacher-Student
• Intrinsic Motivation & Empowerment
• Domain Adaptation & Domain Generalization
• Population-Based Training