Our work uses a compositional approach for precondition learning.
We learn precondition models from a few different example scenes. Each scene is decomposed into its constituent objects and their relations as shown in Figure REF . Learning precondition models from such a structured representation requires us to infer object identities and pairwise object relations.
While object labels can be found using state-of-the-art algorithms [1]}, we focus on learning representations for object pair relations.
Most of the prior work uses discrete representations for object relations [2]}, [3]}, [4]}. These discrete relations are mostly fixed and often insufficient to distinguish between similar scenes.
For instance, in Figure REF , even though both relations might be classified as “above” the semantics of each scene are very different.
Although recent approaches have looked at learning continuous representations for object relations [5]}, [6]},
these approaches use a pre-defined set of relations.
They further assume access to binary labels which allows them to cluster scenes with similar relations together.
<FIGURE>