EE 378A: Course Outline

The following topics will be covered in the course:

  • Statistical decision theory: Bayes, frequentist, estimation, testing, loss function, risk function, sufficiency, completeness, distance between models, minimax theorem, complete class theorem, admissibility, Rao–Blackwell, Cramer–Rao, Lehmann–Scheffe, Maximum likelihood estimator, Bayes estimator, empirical Bayes estimator, James–Stein estimator, density propagation, graphical models, hidden Markov models, belief propagation, deficiency, local asymptotic normality (LAN), Hajek–LeCam convolution and local asymptotic minimax theorem, efficient likelihood estimator, nonparametric statistical methods, bias-variance tradeoff;

  • (Statistical/Individual sequence) learning theory: two cultures of statistical modeling, model function classes instead of probability measures, approximation error–stochastic error tradeoff, analyzing regret instead of risk, VC entropy, VC dimension, VC inequality and its improvements, empirical risk minimization and its converse, structural risk minimization, penalization, Bayesian averaging, ensemble methods, DUDE (Discrete Universal Denoiser), filtering via prediction, S-DUDE, loss estimation and double-sided context trees.