The neuroscience literature, however, suggests that biological visual systems infer and exploit the existence of objects, rather than spatial or temporal blocks with artificial boundaries [1]}, [2]}, [3]}.
Because objects are the atomic units of physical interactions,
it makes sense to discretize on the level of objects.
Numerous object segmentation algorithms have been proposed [4]}, [5]}, [6]}. We chose to use MONet, an unsupervised object segmentation algorithm [7]}. Because MONet is unsupervised, we can train it directly in our domain of interest without the need for object segmentation labels. We emphasize that our choice of MONet is an implementation
detail, and in Appendix , we show that our framework of attention over learned object embeddings also works with other object-segmentation schemes.
We also do not need to place strong demands on the object segmentation algorithm,
e.g. for it to produce aligned output or to have a built-in dynamics model.
<FIGURE>