**Qualitative examples.** We showcase some randomly-selected qualitative examples from Instructions-2M in Table 4. Our inverted prompts are generally on topic and syntactically similar to the originals. Two prompts are perfectly reconstructed. We notice that proper nouns seem difficult; in one example, our model correctly recovers the structure of a question, but mixes up Steinbeck's _Of Mice and Men_ with Salinger's _The Catcher in the Rye_10. (One might assume that this information is represented in the probability distribution, but interestingly the raw probabilities do not favor either title.) In all examples, the system correctly identifies whether or not the prompt ends in punctuation.