We use 2-layer MLPs to parameterize all of the model update functions (node update \(f_v\) , neighborhood aggregation \(f_n\) , agent update \(f_a\) ) and the different inputs are concatenated. We use a Leaky ReLU [1]} activation function with a negative slope of \(0.01\) . As you will see below, we use skip-connections for both node and agent embeddings. This necessitates the use of Layer Normalization (LN) for the MLP inputs. This makes each MLP a Pre-LN residual block [2]}. Both agents and nodes have \(h\) -dimensional embedding vectors.
