We confirm that ELMo's predictions are much closer to the A-but-B rule's manifold than those of the other models by computing KL(q θ ||p θ ) where p θ and q θ are the original and projected distributions: Averaged across all A-but-B sentences and 100 seeds, this gives 0.27, 0.26 and 0.13 for the Kim (2014) , <cite>Hu et al. (2016)</cite> with distillation and ELMo systems respectively.