Contents

Review

Key topics from each chapter

Not exhaustive

Chapter 1

  • Key distributions for categorical data: Poisson, Binomial, Multinomial.

  • Examples of exponential families.

  • Inference for sampling from such families.

Chapter 2

  • Contingency tables.

  • Sampling models: independent rows / columns; Poisson; case-control.

  • Odds ratios.

  • Conditional independence in three-way tables.

Chapter 3

  • Inference for two-way tables.

  • Tests of independence (Pearson / Fisher)

  • Residuals: Pearson and deviance.

  • Ordinal variables.

  • Bayesian models

Chapter 4

  • Generalized Linear Models

  • Basic definitions and examples: binary, counts

  • Likelihood / Score / Hessian (Information)

  • Deviance: likelihood ratio tests

  • Residuals

  • Algorithm: Fisher scoring / Newton-Raphson

Chapter 5

  • Odds ratios

  • Retrospective studies

  • Inference

  • A little bit of inference under misspecification: e.g. bootstrap

  • Multiple logistic regression

Chapter 6

  • Model selection and diagnostics (deviance / Pearson residuals) for logistic regression

  • Analogs of \(R^2\)

  • Mantel-Haenszel tests for stratified 2x2 tables

  • Power in logistic regression

Bayesian GLMs (Touched on in Chapter 7 of Agresti)

  • Brief description of Metropolis-Hastings / Gibbs sampling

  • Simple implementations in STAN

Chapter 8

  • Multinomial regression

  • Multicategory logit

  • Cumulative logit

  • Latent variable representations (data augmentation / Bayesian)

  • Ordinal responses

  • Conditional independence in 3-way tables

Chapter 9

  • Loglinear models

  • Homogeneous association (interactions up to 2-way only)

  • Graphical models

  • Inference (deviance, Wald)

  • Connection to logistic regression

  • Lindsey’s method: density modeling as GLM

Chapter 10

  • Model selection for loglinear models

  • Deviance tests

  • Residuals for diagnostics

Chapter 11

  • Matched pairs and square contingency tables

  • McNemar’s test

  • Subject specific models / conditonal inference

  • Symmetry and quasi-symmetry (Bradley-Terry)

  • Quasi-independence (agreement between raters)

Regularization

  • For GLMs we started off talking about LASSO: $\( {\cal P}(\beta) = \lambda \|\beta\|_1 = \lambda \sum_{j=1}^p |\beta_j|. \)$

  • KKT conditions: solving LASSO yields pair \((\hat{\beta}, \widehat{u}=-\nabla L(\hat{\beta}))\)

Regularization

LASSO picture

Regularization

Solving LASSO: coordinate descent

\[ \hat{\beta}^{(t+1)}_j = \text{argmin}_{\beta_j \in \mathbb{R}} \left[\beta_j \mapsto \Lambda (X_{-j}\hat{\beta}^{(t)}_{-j} + X_j \beta_j) - \beta_j \cdot (X_j^TY) + p_j(\beta_j) \right] \]

  • Sort of analogous to Gibbs sampling but instead of “drawing from dbn” we find its mode.

Regularization

Solving LASSO: proximal gradient

\[ \hat{\beta}^{(t+1)} = S_{\lambda / L}\left( \hat{\beta}^{(t)} - \frac{1}{L} \nabla \ell(\hat{\beta}^{(t)})\right) \]

  • \(S_{\lambda}\) is soft-threshold operator (proximal operator of \(\ell_1\) penalty)

  • Updates all coordinates at once

Survival analysis

  • Lifetime data, often right-censored.

  • Observations are \((O_i, \delta_i)\) with \(O_i=\min(T_i, C_i)\) and $\( \delta_i = \begin{cases} 1 & T_i \leq C_i \\ 0 & \text{otherwise.} \end{cases} \)$

  • Goal: understand distribution of \(T|X\) based on response \((O,C)\).

Survival analysis

Hazard

  • Basic object modeled in survival analysis

  • Presuming density of \(T\) $\( 1 - F(t) = S(t) = \exp(-H(t)), \qquad H(t) = \int_{[0,t]} h(s) \; ds \)$

  • Estimates w/o features: Nelson-Aalen for hazard, Kaplan-Meier for survival

  • Log-rank tests to compare populations

Survival analysis

Accelerated failure time model

\[ S_{\beta}(t;x) = S_0\left(\frac{t}{\theta(x^T\beta)}\right). \]

  • Common choice of link \(\theta(u)=e^{-u}\).

Survival analysis

Proportional hazards model

  • Cox model (fit with partial likelihood) $\( H_{\beta}(t;x) = e^{x^T\beta} H_0(t) \)$

  • Expressible in terms of linear predictor \(\eta_i=X_i^T\beta\).

  • Allows linear modeling / regularization / etc.

  • Various residuals for assessing goodness of fits.

previous

Linear models

next

Assignment 1