For multi-class predictorsSingle-token predictors can be treated as multi-class predictors with the classes being the vocabulary. like language models it is possible to consider the training data as a knowledge base and when retrieving, suggest k nearest neighbours. This can be followed up by averaging the probabilities of the output classes based on the neighbour distance (performed by the aggregator). For this late fusion approach, output gating, the last function \(f_n\)  can either be a simple convex combination \(\hat{y} = \lambda \cdot c + (1-\lambda ) \cdot \xi \)  or a slightly more complex gating mechanism, which computes \(\lambda \)  dynamically based on the current input [1]}, [2]}. Both are discussed in detail in subsec:language-modelling.
