arXiv:2004.07213v2  [cs.CY]  20 Apr 2020Toward Trustworthy AI Development:
Mechanisms for Supporting Veriﬁable Claims∗
Miles Bru
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
With the recent wave of progress in artificial intelligence (AI) has come a
growing awareness of the large-scale impacts of AI systems, and recognition
that existing regulations and norms in industry and academia are insufficient
to ensure responsible AI development. In order for AI developers to earn trust
from system users, customers, civil society, governments, and other
stakeholders that they are building AI responsibly, they will need to make
verifiable claims to which they can be held accountable. Those outside of a
given organization also need effective means of scrutinizing such claims. This
report suggests various steps that different stakeholders can take to improve
the verifiability of claims made about AI systems and their associated
development processes, with a focus on providing evidence about the safety,
security, fairness, and privacy protection of AI systems. We analyze ten
mechanisms for this purpose--spanning institutions, software, and hardware--and
make recommendations aimed at implementing, exploring, or improving those
mechanisms.
Miles Brundage Shahar Avin Jasmine Wang Haydn Belfield Gretchen Krueger Gillian Hadfield Heidy Khlaaf Jingying Yang Helen Toner Ruth Fong Tegan Maharaj Pang Wei Koh Sara Hooker Jade Leung Andrew Trask Emma Bluemke Jonathan Lebensold Cullen O'Keefe Mark Koren Théo Ryffel JB Rubinovitz Tamay Besiroglu Federica Carugati Jack Clark Peter Eckersley Sarah de Haas Maritza Johnson Ben Laurie Alex Ingerman Igor Krawczuk Amanda Askell Rosario Cammarota Andrew Lohn David Krueger Charlotte Stix Peter Henderson Logan Graham Carina Prunkl Bianca Martin Elizabeth Seger Noa Zilberman Seán Ó hÉigeartaigh Frens Kroeger Girish Sastry Rebecca Kagan Adrian Weller Brian Tse Elizabeth Barnes Allan Dafoe Paul Scharre Ariel Herbert-Voss Martijn Rasser Shagun Sodhani Carrick Flynn Thomas Krendl Gilbert Lisa Dyer Saif Khan Yoshua Bengio Markus Anderljung