Which components of the distribution does the inverter need?To investigate whether our model focuses most on the largest components of its input, we iteratively remove (set to the mean) \(k\) components from the probability vector in both ascending and descending order. We also consider removing all but a random subset of \(k\) components from the input. Figure 4 highlights the difference in reconstruction performance across levels of component removal. It appears that the model focuses more on the more likely words. In particular, the smallest \(k\) probabilities are only slightly more useful than a random \(k\). Reconstruction is poor in general until almost the entire distribution is re-included.