Once we learn the object relation model \(f_\text{rel}\)  from pairwise object interaction data we use it for downstream real-world precondition learning tasks.
Our precondition learning model is based on deep neural network architectures which can operate on a set of inputs which are provided by the learned \(f_\text{rel}\)  model.
Formally, given a scene \(S\)  with objects \(S=\lbrace o_1, o_2, \ldots , o_N\rbrace \) . We use \(f_\text{rel}\)  to get object relation embeddings for all \(N\times (N-1)\)  object pairs \(f_\text{rel}(o_i, o_j)\) , where \(o_i\)  and \(o_j\)  are the anchor and referrant object.
For our precondition learning model we use two different network architectures i.e. Relational Networks (RNs) and Graph Neural Networks (GNNs). RNs have been used to reason about objects and their relations [1]}. GNNs are more general neural network architectures
where the input can be represented as a graph \(G = (V, E)\) , with vertices \(V\)  and edges \(E\)  [2]}, [3]}, [4]}.
Figure REF  (Right) shows our formulation of a scene \(S\)  as a GNN. Specifically, we represent each object as a separate node \(v_i \equiv o_i\)  and add bi-directional edges between every node pair. At initialization, we add \(f_\text{rel}(o_i, o_j)\)  as the edge information for edge \(e_{ij} = (v_i, v_j)\) 
Unless explicitly specified, we do not add any node information to the vertices. To allow for better inference, we stack two layers of our graph network architecture. The output of the final graph layer is classified using the sum of all the node and edge embeddings. More details are presented in the supplementary material.
<FIGURE>