Quadratic discriminant analysis (QDA)

Quadratic discriminant analysis (QDA)#

The assumption that the inputs of every class have the same covariance $\mathbf{\Sigma}$ can be quite restrictive.
Bayes boundary ($-- -- --$), LDA ($\cdot\cdot\cdot$), QDA ($--------$).

QDA: multivariate normal with differing covariance#

In quadratic discriminant analysis we estimate a mean $\hat\mu_k$ and a covariance matrix $\hat{\mathbf \Sigma}_k$ for each class separately.
Given an input, it is easy to derive an objective function: $$\delta_k(x) = \log \pi_k - \frac{1}{2}\mu_k^T \mathbf{\Sigma}_k^{-1}\mu_k + x^T \mathbf{\Sigma}_k^{-1}\mu_k - \frac{1}{2}x^T \mathbf{\Sigma}_k^{-1}x -\frac{1}{2}\log |\mathbf{\Sigma}_k|$$
This objective is now quadratic in $x$ and so the decision boundaries are 0s of quadratic functions.