AI Alignment & Safety Reading List

Curated by Mouhssine Rifaki | Stanford Electrical Engineering | Last updated October 2026

As capabilities grow, ensuring systems do what we intend becomes the hard problem. This list gathers concrete problems, scalable oversight ideas, and empirical approaches to building agents that remain corrigible and helpful.

1. Concrete Problems in AI Safety
   Amodei et al. arXiv 2016.
2. Scalable agent alignment via reward modeling: a research direction
   Leike et al. arXiv 2018.
3. Cooperative Inverse Reinforcement Learning
   Hadfield-Menell et al. NeurIPS 2016.
4. Inverse Reward Design
   Hadfield-Menell et al. NeurIPS 2017.
5. Risks from Learned Optimization in Advanced Machine Learning Systems
   Hubinger et al. arXiv 2019.
6. AI safety via debate
   Irving et al. arXiv 2018.
7. AI Safety Gridworlds
   Leike et al. arXiv 2017.
8. The Off-Switch Game
   Hadfield-Menell et al. IJCAI 2017.
9. Supervising strong learners by amplifying weak experts
   Christiano et al. arXiv 2018.
10. Artificial Intelligence, Values and Alignment
    Gabriel. Minds and Machines 2020.