Scaling Laws and Emergent Capabilities Reading List

Curated by Mouhssine Rifaki | Stanford Electrical Engineering | Last updated August 2026

How model performance scales with data, compute, and parameters, and what emerges at scale.

Scaling Laws and Emergent Capabilities: 10 key papers

1. Scaling Laws for Neural Language Models
   Kaplan et al. arXiv 2020.
2. Training Compute-Optimal Large Language Models
   Hoffmann et al. arXiv 2022.
3. Deep Learning Scaling is Predictable, Empirically
   Hestness et al. arXiv 2017.
4. Explaining Neural Scaling Laws
   Bahri et al. arXiv 2021.
5. Scaling Laws for Autoregressive Generative Modeling
   Henighan et al. arXiv 2020.
6. Scaling Laws for Transfer
   Hernandez et al. arXiv 2021.
7. Beyond neural scaling laws: beating power law scaling via data pruning
   Sorscher et al. arXiv 2022.
8. Emergent Abilities of Large Language Models
   Wei et al. arXiv 2022.
9. Are Emergent Abilities of Large Language Models a Mirage?
   Schaeffer et al. arXiv 2023.
10. A Solvable Model of Neural Scaling Laws
    Maloney et al. arXiv 2022.

Related reading lists

• RL from Human Feedback (RLHF)
• Reasoning in LLMs
• Emergent Behavior & Self-Organization
• AI Alignment & Safety
• Model Compression & Pruning
• Communication in Multi-Agent RL
• Vision-Language Models & Multimodal Learning
• Instruction Following & Finetuning