As discussed in Section 3, Grosse et al. (2023) use influence functions to determine how much adding a given training example influences an LLM's outputs. They study auto-regressive pretrained LLMs of up to 52B parameters. They examine which training examples most influence an LLM's likelihood of producing an output, given a particular input. For instance, given the input \(A\), what most influences the likelihood of \(B\)? In their experiments, training examples that match the order ("\(A\) precedes \(B\)")are far more influential than examples with reverse order ("_B_ precedes _A_"). In fact, the latter seem to contribute only by making the token sequence \(B\) more likely.