Stats 300B: Theory of Statistics II

John Duchi, Stanford University, Winter 2018

Course Schedule (subject to change)

Lecture Notes Topics Reading
Tue, Jan 9 Lecture 1 Overview, Convergence of random variables VDV Chapters 2.1, 2.2
Thu, Jan 11 Lecture 2 Convergence of random variables, delta method VDV Chapters 2, 3
Tue, Jan 16 Lecture 3 Asymptotic normality, Fisher information VDV Chapter 5.1-5.6; ELST Chapter 7.1-7.3
Thu, Jan 18 Lecture 4 Fisher information, Moment method VDV Chapter 4; TPE Chapter 2.5
Tue, Jan 23 Lecture 5 Superefficiency, Testing and Confidence Regions ELST Chapter 3.1, 3.2, 4.1; TSH Chapter 12.4
Thu, Jan 25 Lecture 6 Testing: likelihood ratio, Wald, Score tests ELST Chapter 3.1, 3.2, 4.1; TSH Chapter 12.4
Tue, Jan 30 Lecture 7 U-Statistics VDV Chapter 12
Thu, Feb 1 Lecture 8 U-Statistics: Hajek projections and asymptotic normality VDV Chapter 11, 12
Tue, Feb 6 Lecture 9 Uniform laws of large numbers, Covering and Bracketing VDV Chapter 5.2, 19.1, 19.2
Thu, Feb 8 Lecture 10 Subgaussianity, Symmetrization, Rademacher complexity and metric entropy VDV Chapter 19, HDP Chapter 1, 2, 8
Tue, Feb 13 Lecture 11 Symmetrization, Chaining HDP Chapter 8, VDV Chapter 18-19
Thu, Feb 15Lecture 12 Uniform laws via entropy numbers, classes with finite entropy, VC classes VDV Chapter 18-19
Tue, Feb 20Lecture 13 Rademacher complexity and ULLNs VDV Chapter 18-19
Thu, Feb 22Lecture 14 Moduli of continuity, rates of convergence VDV Chapter 18-19
Tue, Feb 27Lecture 15 Weak convergence of random functions VDV Chapter 18-19, Notes on Arzela-Ascoli theorem
Thu, Mar 1Lecture 16 Goodness-of-fit tests, M-estimators with non-differentiable losses VDV Chapter 19.3 & 5.3
Tue, Mar 6 Quadratic-mean differentiability TSH Chapter 12, VDV Chapter 6
Thu, Mar 8Lecture 18 Absolute continuity of measure, Contiguity, LeCams's lemmas, Distance for distributions TSH Chapter 12.3, VDV Chapter 6
Tue, Mar 13Lecture 19 Hellinger distance, Quadratic mean differentiability, Local asymptotic normality, Asymptotically most powerful tests TSH Chapter 12.3, 13.1-13.3, VDV Chapter 6, 7.1-7.3
Thu, Mar 15Lecture 20 Limiting Gaussian experiments, Local asymptotic minimax theorem VDV Chapters 7 and 8, Notes on class website

Note: to get the tex for any of the lectures above, simply click the lecture, and in your browser replace the .pdf extension with .tex. (So, you can get the Lecture 1 pdf and Lecture 1 tex).

  • VDV = van der Vaart (Asymptotic Statistics)

  • HDP = Vershynin (High Dimensional Probability)

  • TSH = Testing Statistical Hypotheses (Lehmann and Romano)

  • TPE = Theory of Point Estimation

  • HDS = Wainwright (High Dimensional Statistics: A Non-Asymptotic Viewpoint)

  • ELST = Elements of Large Sample Theory (Lehmann)

All exercises for the class are available at the 2018 exercise list.

Additional Notes

Topic Link
Arzela-Ascoli Theorem pdf
VC Dimension pdf
Rates of convergence and moduli of continuity pdf
Asymptotics for non-differentiable losses pdf
Contiguity and asymptotics pdf


The scribe notes should be written in prose English, as if in a textbook, so that someone who did not attend the class will understand the material. Please do your best, as it is good practice for communicating with others when you write research papers.

Here is the Scribing Schedule.

All tex files and scribe notes from 2017, 2018, and 2019 are available from their respective syllabi (2017, 2018, 2019). You can download the LaTeX template and style file for scribing lecture notes.