Stats 300B: Theory of Statistics II

John Duchi, Stanford University, Winter 2019

Course Schedule (subject to change)

Lecture Notes Topics Reading
Tue, Jan 8 Lecture 1 Overview, Convergence of random variables VDV Chapters 2.1, 2.2
Thu, Jan 10 Lecture 2 Convergence of random variables, delta method VDV Chapters 2, 3
Tue, Jan 15 Lecture 3 Asymptotic normality, Fisher information VDV Chapter 5.1-5.6; ELST Chapter 7.1-7.3
Thu, Jan 17 Lecture 4 Fisher information, Moment method VDV Chapter 4; TPE Chapter 2.5
Tue, Jan 22 Lecture 5 Superefficiency, Testing and Confidence Regions ELST Chapter 3.1, 3.2, 4.1; TSH Chapter 12.4
Thu, Jan 24 Lecture 6 Testing: likelihood ratio, Wald, Score tests ELST Chapter 3.1, 3.2, 4.1; TSH Chapter 12.4
Tue, Jan 29 Lecture 7 U-Statistics VDV Chapter 12
Thu, Jan 31 Lecture 8 U-Statistics: Hajek projections and asymptotic normality VDV Chapter 11, 12
Tue, Feb 5 Lecture 9 Uniform laws of large numbers, Covering and Bracketing VDV Chapter 5.2, 19.1, 19.2
Thu, Feb 7 Lecture 10 Subgaussianity, Symmetrization, Rademacher complexity and metric entropy VDV Chapter 19, HDP Chapter 1, 2, 8
Tue, Feb 12 Lecture 11 Symmetrization, Chaining HDP Chapter 8, VDV Chapter 18-19
Thu, Feb 14 Lecture 12 Uniform laws via entropy numbers, classes with finite entropy, VC classes VDV Chapter 18-19
Tue, Feb 19 Lecture 13 Rademacher complexity and ULLNs VDV Chapter 18-19
Thu, Feb 21 Lecture 14 Moduli of continuity, rates of convergence, Gaussian sequence model VDV Chapter 18-19, GE Chapter 1
Tue, Feb 26 Lecture 15 Gaussian sequence model, hard and soft thresholding GE Chapter 2
Thu, Feb 28 Lecture 16 Incoherent matrices and concentration inequalities, LASSO HDP Chapter 2-3
Tue, Mar 5 Lecture 17 Lasso and High-dimensional Regression, Generic Chaining HDP 10.5-10.6, HDP 8.5
Thu, Mar 7 Lecture 18 Generic Chaining, Comparison Inequality HDP 8.6, 9.1-9.2
Tue, Mar 12 Lecture 19 Restricted strong convexity and matrix deviation HDP 9.1
Thu, Mar 14 Review

  • VDV = van der Vaart (Asymptotic Statistics)

  • HDP = Vershynin (High Dimensional Probability)

  • TSH = Testing Statistical Hypotheses (Lehmann and Romano)

  • TPE = Theory of Point Estimation (Lehmann)

  • ELST = Elements of Large Sample Theory (Lehmann)

  • GE = Gaussian estimation: Sequence and wavelet models (Johnstone)

Additional Notes

Topic Link
Arzela-Ascoli Theorem pdf
VC Dimension pdf
Rates of convergence and moduli of continuity pdf
Asymptotics for non-differentiable losses pdf
Contiguity and asymptotics pdf


The scribe notes should be written in prose English, as if in a textbook, so that someone who did not attend the class will understand the material. Please do your best, as it is good practice for communicating with others when you write research papers.

Here is the Scribing Schedule.

All tex files and scribe notes from 2018 are available from the 2018 Syllabus. You can download the LaTeX template and style file for scribing lecture notes.