Topics covered in course

Models for discrete data

  • Draw \(X_i \overset{IID}{\sim} F , 1 \leq i \leq N\)

  • Let \(R(X_i), C(X_i)\) be discrete random variables: presence or absence of a trait; labelling a trait.

  • Sample can be summarized in table

\(c_1\)

\(\dots\)

\(c_J\)

Row total

\(r_1\)

\(Y_{11}\)

\(\dots\)

\(Y_{1J}\)

\(Y_{1.}\)

\(\vdots\)

\(\dots\)

\(\dots\)

\(\dots\)

\(\dots\)

\(r_I\)

\(Y_{I1}\)

\(\dots\)

\(Y_{IJ}\)

\(Y_{I.}\)

Column total

\(Y_{.1}\)

\(\dots\)

\(Y_{.J}\)

\(Y_{..}\)


  • Distribution described by

\[ \pi_{ij} = P_F(R=i, C=j), \qquad 1 \leq i \leq I, 1 \leq j \leq J \]

Common questions

  • Independence

\[ \pi_{ij} = \pi_{i.} \cdot \pi_{.j}, \qquad 1 \leq i \leq I, 1 \leq j \leq J \]
  • Homogeneity: when \(I=J\) and values are common

\[ \pi_{i.} = \pi_{.i}, \qquad 1 \leq i \leq I \]

Models:

  • \(Y \sim \text{Multinom}(N, \pi)\)

Alternative:

  • \(N \sim \text{Poisson}(\lambda) \to Y \sim \text{Poisson}(\lambda \cdot \pi)\)

  • That is, \(Y_{ij}\)’s are independent with distribution

\[ Y_{ij} \sim \text{Poisson}(\lambda \pi_{ij}) \]

Regression

Linear regression

  • Response matrix: \(Y \in \mathbb{R}^{n \times q}\)

  • Design matrix: \(X \in \mathbb{R}^{n \times p}\)

Usual model

  • IID draws from

\[ Y_i | X_i \sim N(X_i^T\beta, \sigma^2) \]

Usual estimation problem

\[ \hat{\beta} = \text{argmin}_{\beta} \|Y - X\beta\|^2_2 \]

Binary regression models: \(Y \in \{0,1\}^n\)

Usual model

  • For some CDF \(F\):

\[ Y_i | X_i \sim \text{Binomial}(F(X_i^T\beta)) \]
  • Common choices:

    1. probit: \(F \sim N(0,1)\)

    2. logit: \(F(x) = e^x/(1+e^x)\).

Usual estimation problem

\[ \hat{\beta} = \text{argmin}_{\beta} - 2 \log L(\beta | X, Y) = \text{argmin}_{\beta} \text{DEV}(\beta | X, Y) \]
  • Asymptotic distribution? Inference?


Multinomial: \(Y \in \{1, \dots, L\}^n\)

Some models we’ll see

  • Baseline logistic

  • Ordered logistic


Survival analysis

  • \(T\) a survival time

Basic object: survival function and hazard

\[ P_{\beta}(T > t |X) = \exp \left(-\int_0^t h_{\beta}(s;X) \; ds \right) \]
  • Complications: censoring, truncation.


Non-parameteric methods

  • Kaplan-Meier estimator: direct estimate of \(P(T > t)\) based on IID draws of censored observations from \(F\)

  • Log-rank test

Usual model

  • Cox model:

\[ \frac{h_{\beta}(s;X_1)}{h_{\beta}(s;X_0)} = \exp\left((X_1-X_0)^T\beta \right) \]

Exponential families

Examples

  • Logistic regression

  • Baseline multinomial

  • Poisson

  • Cox model

Tools for exponential families

  • EM algorithm

  • Gibbs sampling

  • Pseudo-likelihood