The probability tables from the last section, e.g., \(\mathbb {P}(Y_{s, p, o, t})\)  and
\(\mathbb {P}(s, o, t)\) 
can be represented by tensors. In many application domains, one works with reduced-rank tensor factorization models, e.g., to reduce parameter size, computational complexity, and generalization [1]}.
In standard approaches, tensor products of index embedding vectors
are used, as in the Tucker model (a.k.a. tensor subspace approximation) or RESCAL [2]}. In the canonical polyadic decomposition (CPD) and the tensor train models (a.k.a. matrix product states) only a small number of elements in the tensor product are considered and one achieves approximations with complexity linear in the order of the tensor [1]}. Another way to reduce complexity is to employ neural networks, replacing the tensor products and leading to generalized tensor models.
These approaches are known as Neural Tensor Networks [4]}, E-MLPs, ER-MLPs [5]}, [6]}, and multiway neural networks [7]}.
Our Bilayer Tensor Network (BTN) is another instance, where a recurrent neural network and an attention mechanism are employed.
