Abstract

In many applications, the objective of variable selection is to find a good compromise between the likelihood and the complexity of the model. The balance between the likelihood and complexity is controlled by a regularisation parameter. The selection and estimation thus proceeds in two stages. The first step is the assessment of the regularisation parameter, through the optimisation of an information criterion, which estimates the distance to the true model. The second step is then to find the best selection and estimation for a given value of the regularisation parameter. Obviously, the former step has to anticipate for effects occurring during the latter step. In particular, when the class of models under investigation is high-dimensional while the true model is sparse, then a relatively large number of false positives may contribute to the likelihood. The impact of false positive selections on the likelihood can be reduced by shrinking the estimates, especially the smaller ones. This approach, however, makes the selection procedure as a whole too tolerant for false positives, leading to a major overestimation of the model size. If we take the model size as complexity measure, then the best estimation within a selection involves no shrinkage. The effect of false positives can then be described as a so-called mirror: among the parameters that are not prominently part of the true model, false positives present themselves as the best candidates for being part of the model, whereas in reality, they are worse than a random choice of a non-significant parameter. We present information criteria that adjust for this mirror effect.