Math 275/Stat 375 – Mathematical problems in Machine Learning

Andrea Montanari, Stanford University, Spring 2024

This class provides an introduction to a certain number of theoretical ideas that have been developed with the objective of understanding modern deep learning methods. Specific topics might include

Empirical risk minimization and empirical process theory:

  • Uniform convergence guarantees

  • Complexity bounds for neural networks

Optimization and implicit regularization:

  • Linear models

  • Nonlinear models

Generalization in the linear regime:

  • The interpolation phase transition

  • Random features models

  • Neural tangent models

Feature learning:

  • Mean field theory

  • Other parametrizations

  • Sparse and multi-index models

Diffusion models

  • Guarantees

  • Connections with stochastic localization

Attentions and transformers:

  • Examples of in-context learning

Learning out of distribution

Class Times and Location

  • Tue-Thu, 9:00-10:20AM

  • Sequoia 200

Office hours

  • Thu, 2:30-3:30PM (Starting on 4/18)

  • Sequoia 133


First lecture on Tuesday, April 2

No lecture on April 11