Most often in deep learning, ensembling simply averages the outputs of \(n\)  models that differ in their initialization [1]}. This ensembling is commonly motivated by the desire to reduce the variance of the predictive model. Indeed, for the MSE, we know that ensembling by averaging the outputs of models drawn in i.i.d. fashion (a) reduces the variance, and (b) conserves the bias.
