Using ghost variables to interpret complex predictive models

Abstract

We provide a novel method to determine the degree to which each explanatory variable in a sophisticated prediction model is relevant, as framed in the literature on interpretable machine learning. We presum that we have a test set to evaluate the model's performance outside of the sample and a training set to fit the model. By comparing the model's predictions in the test set with those obtained when the variable of interest is replaced (in the test set) by its ghost variable, which is defined as the prediction of this variable using the other explanatory variables, we propose to measure the individual relevance of each variable.