Learning Representations of Sets through Optimized Permutations
Representations of sets are challenging to learn because operations on sets should be permutation invariant. To this end, we propose a Permutation Optimisation module that learns how to permute a set end to end. The permuted set can be further processed to learn a permutation invariant representation of that set, avoiding a bottleneck in traditional set models. We demonstrate our model's ability to learn permutations and set representations with either explicit or implicit supervision on four datasets, on which we achieve state of the art results: number sorting, image mosaics, classification from image mosaics, and visual question answering.
ICLR