Hyperparameter Optimization & AutoML Reading List

Curated by Mouhssine Rifaki | Stanford Electrical Engineering | Last updated August 2026

Essential papers on hyperparameter optimization, Bayesian optimization, and automated machine learning.

Hyperparameter Optimization & AutoML: 10 key papers

1. Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization
   Li et al. arXiv 2016.
2. BOHB: Robust and Efficient Hyperparameter Optimization at Scale
   Falkner et al. arXiv 2018.
3. Optuna: A Next-generation Hyperparameter Optimization Framework
   Akiba et al. arXiv 2019.
4. Hyperparameter Optimization: A Spectral Approach
   Hazan et al. arXiv 2017.
5. Gradient-based Hyperparameter Optimization through Reversible Learning
   Maclaurin et al. arXiv 2015.
6. Forward and Reverse Gradient-Based Hyperparameter Optimization
   Franceschi et al. arXiv 2017.
7. Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions
   MacKay et al. arXiv 2019.
8. Hyperparameters in Reinforcement Learning and How To Tune Them
   Eimer et al. arXiv 2023.
9. Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges
   Bischl et al. arXiv 2021.
10. Learning to learn by gradient descent by gradient descent
    Andrychowicz et al. arXiv 2016.

Related reading lists

• Population-Based Training
• Policy Gradient Methods
• Bayesian Optimization & Global Optimization
• Spectral Methods in Machine Learning
• Optimization for Deep Learning
• Multi-Task & Transfer Reinforcement Learning
• In-Context RL & Prompting
• Causal Discovery & Structure Learning