In a model-agnostic setting, a natural definition of VI that is independent of the estimation procedure is to measure the loss of predictive power when the variables of interest are deleted. To estimate such model-agnostic VI, retraining is the most widely used type of method, which involves training separate models on the reduced data with the variables of interest deleted and assessing the predictive skill difference [1]}, [2]}, [3]}. Retraining often acts as the best benchmark to evaluate other VI estimation methods [4]} due to its accuracy, yet it is computationally infeasible in high-dimensional settings.
Other methods for VI estimation include knockoff methods [5]}, [6]} and Floodgate [7]}, which require the co-variate distribution to be known.
An alternative approach is to use a dropout-type method [8]}. Dropout is best-suited for assessing how much a variable affects a predictive model, as opposed to our goal of assessing how much a variable affects the response. Despite the resulting issues with VI estimation accuracy, it is still widely used in practice as a proxy for VI due to its computational tractability.
