Models.We train models to invert the distribution of Llama-2 (7B) and Llama-2 Chat (7B) (Touvron et al., 2023). We choose this model because, as of the time of publication, it is the best-performing open-source LLM at the 7B parameter scale. We assume full access to the model output probabilities except for during the distribution extraction experiments in Section 7, in which we set temperature to \(0\) and provide a single logit bias argument.