The v0.5 Benchmark only tests for whether models give unsafe responses. An additional concern is whether models refuse to engage with safe prompts, which is also known as "false refusal" [31]. This is a problem, as it reduces free use of models and can lead to censorship and restrictions on free expression. Because all of the prompts associated with the typical adult user persona are benign, we have a set of test items that could be used to test false refusal. However, this is not included in v0.5, as we do not have the resources to actually assess whether models' responses constitute a false refusal.