EE278: Course Plan

Stanford University, Ayfer Özgür and Tsachy Weissman, Fall 2026

Fall 2026 Course Topics and Readings

Suggested textbook sections are listed by topic, in the order of the lecture plan below. The readings are organized by topic rather than lecture date because the schedule is tentative.

Section numbers refer to Gallager (1st edition), Vershynin (1st edition), and Understanding Machine Learning (UML). Related readings provide background rather than full coverage. Where no direct textbook section is listed, use the lecture material, including recordings in Canvas and notes on Ed.

  • Inference; Hypothesis Testing; MAP and ML
    Gallager, §§8.1–8.2

  • Gaussian Hypothesis Testing; Error Probability
    Gallager, §§8.2.2–8.2.3 (Gaussian detection), §8.4 (error curves)

  • Minimum-Distance Decoding
    Related reading: Gallager, §8.5 (multiple-hypothesis detection); see lecture material for minimum-distance decoding

  • Detection with Multiple Observations
    Gallager, §8.2.3 (vector observations)

  • Machine Learning; Hypothesis Classes and Loss Functions; Empirical Risk Minimization
    UML, §§2.1–2.3

  • Generalization Gap
    UML, §2.3.1

  • Weak LLN
    Gallager, §1.7.1

  • Markov's Inequality; Chebyshev's Inequality
    Gallager, §§1.6.1–1.6.2; Vershynin, §1.2

  • CLT
    Gallager, §1.7.3; Vershynin, §1.3

  • Hoeffding's Inequality
    Vershynin, §2.2; UML, Appendix B.4

  • Generalization Bounds; Uniform Convergence
    UML, §2.3.1 and §§4.1–4.2

  • Approximation Error and Excess Risk
    UML, §5.2 (error decomposition; excess risk is termed estimation error)

  • Random Vectors; Covariance Matrices; Linear Transforms of Random Vectors
    Related reading: Vershynin, §3.2 (covariance and linear transformations); Gallager, §§3.3–3.4 (Gaussian examples and covariance)

  • Gaussian Random Vectors; Gaussian Densities and Degeneracy; Affine Transformations of Gaussian Vectors; Independence of Gaussian Components
    Gallager, §§3.3–3.4; Vershynin, §3.3.2

  • Eigenvalues and Eigenvectors; Spectral Decomposition
    Gallager, §3.4.1; Vershynin, §3.2

  • Principal Component Analysis; Dimensionality Reduction
    UML, §23.1; Vershynin, §3.2.1 (PCA background)

  • Expectation, Variance, and Covariance; Correlation
    Related reading: Gallager, §1.5 (expectation and probability review)

  • Conditional Expectation; Law of Iterated Expectations
    Related reading: Gallager, §1.5.2 (conditional expectation)

  • Conditional Variance; Law of Total Variance
    Lecture material

  • MMSE Estimation
    Gallager, §10.1.1 (squared-error estimation)

  • Properties of the MMSE Estimator; Inner Product Spaces of Random Variables; Orthogonal Projection; Orthogonality Principle
    Gallager, §10.1.1 (squared-error estimation), §10.6 (geometry and orthogonality)

  • Linear MMSE Estimation
    Gallager, §10.3

  • Vector Linear MMSE Estimation
    Gallager, §10.3 (linear MMSE estimation); related reading: §10.4 (vector signal estimation)

  • Error Covariance Matrices
    Related reading: Gallager, §10.4 (vector signal estimation); see lecture material

  • Gaussian Conditioning
    Gallager, §3.5

  • Gaussian Estimation
    Gallager, §10.2

  • Innovations
    Related reading: Gallager, §§10.2.1, 10.4.3 (iterative estimation); see lecture material for innovations

  • State-Space Models
    Related reading: Gallager, §10.2.2 and §10.4.4 (Kalman filter models)

  • Kalman Filtering; State Prediction and Filtering; Kalman Gain; Error Covariance Recursions
    Gallager, §10.2.2 (scalar), §10.4.4 (vector)

  • Random Processes; IID Processes
    Gallager, §1.4 and the opening of §3.6 (random processes); related reading: §1.4.1 (Bernoulli processes as an IID example)

  • Random Walks
    Gallager, §9.1 (random walks)

  • Markov Processes
    Related reading: Gallager, §4.1 (finite-state Markov chains), §7.1 (countable-state Markov processes)

  • Processes with Independent Increments
    Related reading: Gallager, §2.2 (Poisson-process example); see lecture material for the general definition

  • Counting Processes
    Related reading: Gallager, §2.1.1 (arrival processes)

  • Poisson Limit Theorem
    Related reading: Gallager, §2.2.5 (Poisson limits of Bernoulli processes); see lecture material for the binomial-to-Poisson limit

  • Poisson Processes
    Gallager, §2.2

  • Mean and Autocorrelation Functions
    Related reading: Gallager, the opening of §3.6 and §3.6.1 (means and covariance functions); see lecture material for autocorrelation functions

  • Stationarity
    Gallager, §3.6.1

  • Gaussian Processes
    Gallager, the opening of §3.6

  • Wide-Sense Stationarity (WSS)
    Gallager, §3.6.1

  • Power Spectral Density
    Gallager, §§3.6.6–3.6.7

  • Cross-Correlation
    Lecture material

  • Cross-Spectral Density
    Lecture material

  • Linear Systems with Random Process Inputs
    Gallager, §§3.6.6–3.6.7

  • White Noise
    Gallager, §3.6.8 (white Gaussian noise)

  • Linear MMSE Estimation of Random Processes
    Related reading: Gallager, §10.3 (finite-vector estimation)

  • Wiener Filtering; Spectral Factorization; Causal Wiener Filtering
    Lecture material

  • Lookahead; Linear Prediction; Prediction in Practice
    Lecture material

Fall 2026 Tentative Lecture Plan

Lecture topics, pacing, and teaching assignments are tentative and subject to change. Lectures meet on Tuesdays and Thursdays. Lecture numbers include the midterms and Democracy Day. University holidays and recess dates follow the Stanford academic calendar.

  • Lecture 1 (September 22): Inference; Hypothesis Testing; MAP and ML — Professor Ayfer Özgür

  • Lecture 2 (September 24): Gaussian Hypothesis Testing; Error Probability; Minimum-Distance Decoding; Detection with Multiple Observations — Professor Ayfer Özgür

  • Lecture 3 (September 29): Machine Learning; Hypothesis Classes and Loss Functions; Empirical Risk Minimization; Generalization Gap; Weak LLN — Professor Ayfer Özgür

  • Lecture 4 (October 1): Markov's Inequality; Chebyshev's Inequality; Weak LLN; CLT; Hoeffding's Inequality; Generalization Bounds; Uniform Convergence — Professor Ayfer Özgür

  • Lecture 5 (October 6): Uniform Convergence; Generalization Bounds; Approximation Error and Excess Risk; Random Vectors; Covariance Matrices; Linear Transforms of Random Vectors — Professor Ayfer Özgür

  • Lecture 6 (October 8): Gaussian Random Vectors; Gaussian Densities and Degeneracy; Affine Transformations of Gaussian Vectors; Independence of Gaussian Components — Professor Ayfer Özgür

  • Lecture 7 (October 13): Covariance Matrices; Eigenvalues and Eigenvectors; Spectral Decomposition; Principal Component Analysis; Dimensionality Reduction — Professor Ayfer Özgür

  • Lecture 8 (October 15): Midterm I (in class during class time; covers the first three weeks)

  • Lecture 9 (October 20): Expectation, Variance, and Covariance; Correlation; Conditional Expectation; Law of Iterated Expectations; Conditional Variance; Law of Total Variance; MMSE Estimation — Professor Tsachy Weissman

  • Lecture 10 (October 22): Properties of the MMSE Estimator; Inner Product Spaces of Random Variables; Orthogonal Projection; Orthogonality Principle; Linear MMSE Estimation — Professor Tsachy Weissman

  • Lecture 11 (October 27): Vector Linear MMSE Estimation — Professor Tsachy Weissman

  • Lecture 12 (October 29): Vector Linear MMSE Estimation; Error Covariance Matrices; Gaussian Conditioning; Gaussian Estimation; Innovations; State-Space Models — Professor Tsachy Weissman

  • Lecture 13 (November 3): Democracy Day (no classes)

  • Lecture 14 (November 5): Midterm II (in class during class time; covers Lectures 1–12, with emphasis on Lectures 7–12)

  • Lecture 15 (November 10): Kalman Filtering; State Prediction and Filtering; Innovations; Kalman Gain; Error Covariance Recursions — Professor Tsachy Weissman

  • Lecture 16 (November 12): Random Processes; IID Processes; Random Walks; Markov Processes; Processes with Independent Increments; Counting Processes — Professor Tsachy Weissman

  • Lecture 17 (November 17): Poisson Limit Theorem; Poisson Processes; Mean and Autocorrelation Functions; Stationarity — Professor Tsachy Weissman

  • Lecture 18 (November 19): Gaussian Processes; Wide-Sense Stationarity (WSS); Power Spectral Density — Professor Tsachy Weissman

  • Lecture 19 (December 1): Cross-Correlation; Cross-Spectral Density; Linear Systems with Random Process Inputs; White Noise; Linear MMSE Estimation of Random Processes; Wiener Filtering — Professor Tsachy Weissman

  • Further Topics (Schedule to Be Determined): Spectral Factorization; Causal Wiener Filtering; Lookahead; Linear Prediction

  • Lecture 20 (December 3): Course Wrap-Up and Prediction in Practice — Professor Ayfer Özgür