Fall 2026 Course Topics and Readings
Suggested textbook sections are listed by topic, in the order of the lecture plan below. The readings are organized by topic rather than lecture date because the schedule is tentative.
Section numbers refer to Gallager (1st edition), Vershynin (1st edition), and Understanding Machine Learning (UML). Related readings provide background rather than full coverage. Where no direct textbook section is listed, use the lecture material, including recordings in Canvas and notes on Ed.
Inference; Hypothesis Testing; MAP and ML
Gallager, §§8.1–8.2 Gaussian Hypothesis Testing; Error Probability
Gallager, §§8.2.2–8.2.3 (Gaussian detection), §8.4 (error curves) Minimum-Distance Decoding
Related reading: Gallager, §8.5 (multiple-hypothesis detection); see lecture material for minimum-distance decoding Machine Learning; Empirical Risk Minimization
UML, §§2.1–2.3 Weak LLN
Gallager, §1.7.1 Markov's Inequality; Chebyshev's Inequality
Gallager, §§1.6.1–1.6.2; Vershynin, §1.2 CLT
Gallager, §1.7.3; Vershynin, §1.3 Hoeffding's Inequality
Vershynin, §2.2; UML, Appendix B.4 Generalization Bounds; Uniform Convergence
UML, §2.3.1 and §§4.1–4.2 MMSE Estimation; Geometric Interpretation; Orthogonality Principle
Gallager, §10.1.1 (squared-error estimation), §10.6 (geometry and orthogonality) Gaussian Estimation
Gallager, §10.2 Random Vectors; Covariance Matrices; Gaussian Random Vectors
Gallager, §§3.3–3.4; Vershynin, §§3.2, 3.3.2 Linear MMSE Estimation
Gallager, §10.3 Principal Component Analysis; Optimal Linear Approximation
UML, §23.1; Vershynin, §3.2.1 (PCA background) Gaussian Conditioning
Gallager, §3.5 Innovations
Related reading: Gallager, §§10.2.1, 10.4.3 (iterative estimation); see lecture material for innovations Kalman Filtering
Gallager, §10.2.2 (scalar), §10.4.4 (vector) Random Processes; Stationarity
Gallager, §1.4 and the opening of §3.6 (random processes), §3.6.1 (stationarity) Power Spectral Density; Linear Systems with Random Process Inputs
Gallager, §§3.6.6–3.6.7 Cross-Correlation; Cross-Spectral Density
Lecture material White Noise
Gallager, §3.6.8 (white Gaussian noise) Linear Estimation
Gallager, §10.3 (finite-vector estimation) Wiener Filtering; Spectral Factorization; Causal Wiener Filtering
Lecture material Lookahead; Linear Prediction; Prediction in Practice
Lecture material
Fall 2026 Tentative Lecture Plan
Lecture topics, pacing, and teaching assignments are tentative and subject to change. Lectures meet on Tuesdays and Thursdays. Lecture numbers include the midterms and Democracy Day. University holidays and recess dates follow the Stanford academic calendar.
Lecture 1 (September 22): Inference; Hypothesis Testing; MAP and ML — Professor Ayfer Özgür Lecture 2 (September 24): Gaussian Hypothesis Testing; Error Probability; Minimum-Distance Decoding — Professor Ayfer Özgür Lecture 3 (September 29): Machine Learning; Empirical Risk Minimization; Weak LLN — Professor Ayfer Özgür Lecture 4 (October 1): Markov's Inequality; Chebyshev's Inequality; CLT; Hoeffding's Inequality; Generalization Bounds; Introduction to Uniform Convergence — Professor Ayfer Özgür Lecture 5 (October 6): Uniform Convergence; MMSE Estimation; Geometric Interpretation — Professor Ayfer Özgür Lecture 6 (October 8): Orthogonality Principle; Gaussian Estimation; Random Vectors; Covariance Matrices — Professor Ayfer Özgür Lecture 7 (October 13): Midterm I (in class during class time) Lecture 8 (October 15): Gaussian Random Vectors; Linear MMSE Estimation — Professor Ayfer Özgür Lecture 9 (October 20): Principal Component Analysis; Optimal Linear Approximation — Professor Tsachy Weissman Lecture 10 (October 22): Gaussian Conditioning; Innovations — Professor Tsachy Weissman Lecture 11 (October 27): Kalman Filtering — Professor Tsachy Weissman Lecture 12 (October 29): Random Processes — Professor Tsachy Weissman Lecture 13 (November 3): Democracy Day (no classes) Lecture 14 (November 5): Midterm II (in class during class time) Lecture 15 (November 10): Stationarity; Power Spectral Density — Professor Tsachy Weissman Lecture 16 (November 12): Linear Systems with Random Process Inputs; Cross-Correlation; Cross-Spectral Density — Professor Tsachy Weissman Lecture 17 (November 17): White Noise; Linear Estimation; Wiener Filtering — Professor Tsachy Weissman Lecture 18 (November 19): Spectral Factorization; Causal Wiener Filtering — Professor Tsachy Weissman Lecture 19 (December 1): Lookahead; Linear Prediction — Professor Tsachy Weissman Lecture 20 (December 3): Course Wrap-Up and Prediction in Practice — Professor Ayfer Özgür
|