The v1.0 benchmark will provide meaningful insights into the safety of AI systems. However, **the v0.5 benchmark should not be used to assess the safety of AI systems**. We have released it only to outline our approach to benchmarking, and to solicit feedback. For this reason, all the models we tested have been anonymized. We have sought to fully document the limitations, flaws, and challenges of the v0.5 benchmark in this paper, and we are actively looking for input from the community.