More recent work has addressed this challenge by 'distilling' the models, training smaller versions of <cite>BERT</cite> which reduce the number of parameters to train by 40% while retaining more than 95% of the full model performance and even outperforming it on two out of eleven GLUE tasks .