Basis expansions#

Fig 7.1

Problem: How do we model a non-linear relationship?

  • Left: Regression of wage onto age.

  • Right: Logistic regression for classes wage>250 and wage<250

Strategy:#

  • Define a model:

\[Y = \beta_0 + \beta_1 f_1(X) + \beta_2 f_2(X) + \dots + \beta_d f_d(X) + \epsilon.\]

  • Fit this model through least-squares regression: \(f_j\)’s are nonlinear, model is linear!

  • Some options for \(f_1,\dots,f_d\):

    1. Polynomials, \(f_i(x) = x^i\).

    2. Indicator functions, \(f_i(x) = \mathbf{1}(c_i \leq x < c_{i+1})\).

Piecewise constant functions#

Fig 7.2

Piecewise polynomial functions#

Fig 7.3