Different open-source and proprietary LLMs offer APIs with varying capabilities, which impact the ability to perform model extraction attacks and the choice of attack algorithm. A summary of the different APIs we study, and our motivation for doing so, is presented in Table 1. The logits API is a strawman threat model where the API provides logits for all tokens in the response to a given prompt. We begin with this toy setting, as the attack techniques we develop here can be reused in subsequent sections, where we will first reconstruct the logits from more limited information (e.g., log-probabilities for only the top few tokens) and then run the attack.