Our conclusion seems to be distinct to to the existing work [1]}, which claims that the ConcatMLP is hard to learn a Dot Product.
Actually, our conclusion is not conflicted with that in [1]}.
This ConcatMLP decoder processes the concatenation of the paired embeddings instead of the Hadamard product of the paired embeddings as the HadamardMLP.
The HadamardMLP holds the inductive bias similar to the Dot Product, which makes the former easily learns the latter.
Actually, we show that a simple two-layer MLP with only two hidden units is equivalent to the Dot Product with specific weights.
We assign the first layer weights for two hidden units as \(\mathbf {1}\)  and \(-\mathbf {1}\)  and the second layer weights as ones.
Then, we have its output as:
\(s_{ij} = \phi ^{\mathrm {MLP}}(\mathbf {x_i}, \mathbf {x}_j) = \mathrm {ReLU}(\mathbf {1} \mathbin {\hbox{\scalebox {.7}{$\bullet $}}}\) 
