Article
Influence of single observations on the choice of the penalty parameter in ridge regression
Search Medline for
Authors
-
Kristoffer Hellton
- Norwegian Computing Center, Oslo, Norway
-
Camilla Lingjærde
- Norwegian Computing Center, Oslo, Norway
-
Riccardo De Bin
- University of Oslo, Oslo, Norway
| Published: | February 26, 2021 |
|---|
Outline
Text
Penalized regression methods, such as ridge regression, heavily rely on the choice of a tuning, or penalty, parameter, which is often computed via cross-validation. Discrepancies in the value of the penalty parameter may lead to substantial differences in regression coefficient estimates and predictions. In this paper, we investigate the effect of single observations on the optimal choice of the tuning parameter, showing how the presence of influential points can dramatically change it. We distinguish between points as “expanders” and “shrinkers”, based on their effect on the model complexity. Our approach supplies a visual exploratory tool to identify influential points, naturally implementable for high-dimensional data where traditional approaches usually fail. Applications to real data examples, both low- and high-dimensional, and a simulation study are presented.
The authors declare that they have no competing interests.
The authors declare that an ethics committee vote is not required.
