Given our scope (see above), we only evaluate **AI systems that use chat-tuned LMs**. These are foundation models that have been fine-tuned to engage in dialogues and follow instructions through an alignment algorithm, such as reinforcement learning with human feedback [65], reinforcement learning with AI feedback [66], and supervised fine-tuning [67]. We only tested open models with permissive licenses against the v0.5 benchmark.