Is 'Which model ...' the right question?


This paper presents a critique of the standard two-stage approach in which a well-fitting model is identified first, and then applied in one or several subsequent inferences. Although deeply ingrained in practice, this approach is deficient because the possibility of an erroneous decision in the model selection (model uncertainty) is discounted. The criticism applies to all model selection procedures, equally in the frequentist and Bayesian perspectives. We describe an approach based on synthetic estimation, in which the estimators based on the candidate models are linearly combined. Its properties are explored in the setting of an experiment from information technology.