Our first approach relies on a common "reference" token that lets us learn the relative difference between all logits (this is the best we can hope for, since the softmax is invariant under additive shifts to the logits). Suppose the top token for a prompt is \(R\), and we want to learn the relative difference between the logits of tokens \(i\) and \(R\). We add a large bias \(B\) to token \(i\) to push it to the top-5, and then observe the logprobs of both token \(i\) and \(R\).