To perform the experiment illustrated in Figure 2, we sample 100 paragraphs from Wikipedia obtained via the Wikitext dataset (Merity et al., 2016). We prompt GPT-4 with the first ten words of each paragraph prepended by the text _"Please update the sentence by replacing one word sentence with a close synonym. Respond with only the word to swap in the format word1 -? word2."_. We then extract the word swap from GPT-4's response and apply it to the input to produce the transformed input \(\hat{x}\). The language model used for prompting is the 7-billion parameter version of LLAMA-2 (non-chat version).