To guide practitioners, we have created a test specification for the AI Safety Benchmark, and have made it freely available.9 The test specification was created and vetted by a large group of researchers and practitioners in the WG. Its creation was motivated by ongoing challenges around the integrity of performance results and their sensitivity to seemingly small setup changes, such as prompt formulation, few-shot learning configurations, and chain-of-thought instructions. If these factors and configuration parameters are not well-documented, this can lead to seemingly inexplicable variations in SUTs' performance and limit reproducibility. Our test specification can help practitioners in two ways. First, it can aid test writers to document proper usage of a proposed test and enable scalable reproducibility amongst a large group of stakeholders who may want to either implement or execute the test. Second, the specification schema can also help audiences of test results to better understand how those results were created in the first place. We aim to produce more specification resources in the future.