Assumptions of our threat model.We motivate this problem by the prevalence of language models as a service. In these use cases we assume that an unknown prefix sequence, known as the _prompt_, is prepended to the user input. We consider varying levels of model access: full distributional access, partial distributional access (top-K or by request), text output with user-defined logit bias, and text output access only. We assume no access to the model weights or activations.