**Out-of-domain.** Table 2 shows the results when using prompts that are significantly different than the training distribution both in length and in content. For these domains, the model is significantly better than few-shot and jailbreaking on the RLHF model. With the chat model, we also observe that the jailbreak strings are especially ineffective on the Anthropic HH dataset, which contains a large amount of toxic content; this indicates that the chat model is less likely to obey the user's request when toxic content is present in the prompt. For the raw model, the inversion approach is a bit worse than jailbreaking on BLEU.