Non-asymptotic tests of model performance

This paper describes a non-asymptotic approach to the problem of selection bias in economic forecasting. By using non-asymptotic measure concentration results, it is possible to deal with settings in which the class of potential models is large with respect to the number of data points. The bounds on p values obtained by these methods are necessarily conservative, but they provide a useful benchmark for model selection in settings where asymptotics may not apply.