Stats 300B: Theory of Statistics II

John Duchi, Stanford University, Winter 2019


Tuesday and Thursday, 10:30 - 11:50 AM in Green Earth Sciences, Room 150.

Course Staff Email Address

stats300b-win1819-staff_albatross_lists_duck_stanford_duck_edu (replace _albatross_ with @ and _duck_ with .)


John Duchi

  • Office hours: Tuesdays and Thursdays, 3:00pm - 4:00pm, 126 Sequoia Hall.

Teaching Assistants

Stephen Bates

  • Office hours: Wednesday 4:00pm - 6:00pm, 220 Sequoia Hall.

Elena Tuzhilina

  • Office hours: Mondays 10:00am -12:00pm, 420-286.


Stats 300A, Stats 310A.





Your grade will be determined by scribing (5%), weekly problem sets (60%), and a final exam (35%).

There will be weekly homework assignments throughout the course, which will count for 60% of the grade. In effort to speed grading and homework return to you, we will grade homework problems and their sub-parts on a {0, 1, 2}-scale: 0 indicates a completely incorrect answer, 1 indicating approximately halfway correct, 2 indicating more or less correct. Everyone will be required to scribe at least one lecture (5% of final grade), and (if necessary because of enrollment) we need some students to scribe two lectures, an additional scribed lecture will increase the percentage score S of your lowest homework to min{100, S + 50} (that is, by 50%). There will be a final exam (either in class or take home, TBD by preference of class) worth 35% of the grade.

Course Overview

In this class, we will cover classical and not-so-classical techniques underpinning asymptotic statistics and large sample theory. We will highlight a few of the big ideas, but the breadth of the subject by now is so large that our treatment will necessarily be spotty.


Maximum likelihood estimation and models (2 weeks)

Basic M-estimators, Asymptotic normality via the central limit theorem (CLT), Taylor-like models, The delta-method

Testing (2 weeks)

Confidence intervals and p-values, Relative efficiency, Nonparametric tests, Worst-case alternatives

U-statistics (1 week)

Projections, Hoeffding and Hajek decompositions, One-sample and two-sample U-statistics, Degenerate U-statistics

Convergence of general random variables, processes, and uniform laws (3 weeks)

Basic concentration inequalities, Uniform laws of large numbers and Glivenko Cantelli classes, Rademacher complexity, symmetrization, matrix concentration, Prohorov and Portmanteau theorems, Uniform central limit theorems

Optimality, power, and contiguity (2 weeks)

Contiguity and absolute continuity, Local asymptotic normality and quadratic mean differentiability, Limiting normal experiments and asymptotic power, Local asymptotic minimax theorems, Super-efficienty and the Hodges phenomenon