Building Models Shrinkage Methods
Shrinkage - What Do I Use It For?
Shrinking extreme values towards a central value results in a better estimate
of the true mean.
Why?
More stable parameter estimates
(less extreme outliers considered)
Reduction of sampling and
non-sampling errors
Disadvantages
Erroneous estimates if population
has atypical mean. Knowing when
this is the case is difficult.
Possible introduction of bias.
Shrunk models may fit new data
worse than original models would.
How?
Fitting a model with all
p
predictors
Shrink estimated coefficients
towards zero relative to the least
squares estimates
Depending on what type of shrinkage is
performed, some of the coefficients
may be estimated to be exactly zero.
Hence, shrinkage methods can also
perform variable selection.
Aarhus University Biostatistics - Why? What? How? 21 / 33