TITLE: Measuring Ethics in AI with AI: A Methodology and Dataset Construction 

ABSTRACT: Recently, the use of sound measures and metrics in Artificial Intelligence
has become the subject of interest of academia, government, and industry.
Efforts towards measuring different phenomena have gained traction in the AI
community, as illustrated by the publication of several influential field
reports and policy documents. These metrics are designed to help decision
takers to inform themselves about the fast-moving and impacting influences of
key advances in Artificial Intelligence in general and Machine Learning in
particular. In this paper we propose to use such newfound capabilities of AI
technologies to augment our AI measuring capabilities. We do so by training a
model to classify publications related to ethical issues and concerns. In our
methodology we use an expert, manually curated dataset as the training set and
then evaluate a large set of research papers. Finally, we highlight the
implications of AI metrics, in particular their contribution towards developing
trustful and fair AI-based tools and technologies. Keywords: AI Ethics; AI
Fairness; AI Measurement. Ethics in Computer Science.
Recently, the use of sound measures and metrics in Artificial Intelligence
has become the subject of interest of academia, government, and industry.
Efforts towards measuring different phenomena have gained traction in the AI
community, as illustrated by the publication of several influential field
reports and policy documents. These metrics are designed to help decision
takers to inform themselves about the fast-moving and impacting influences of
key advances in Artificial Intelligence in general and Machine Learning in
particular. In this paper we propose to use such newfound capabilities of AI
technologies to augment our AI measuring capabilities. We do so by training a
model to classify publications related to ethical issues and concerns. In our
methodology we use an expert, manually curated dataset as the training set and
then evaluate a large set of research papers. Finally, we highlight the
implications of AI metrics, in particular their contribution towards developing
trustful and fair AI-based tools and technologies. Keywords: AI Ethics; AI
Fairness; AI Measurement. Ethics in Computer Science.
Measuring Ethics in AI with AI: A Methodology and Dataset Construction
http://arxiv.org/abs/2107.11913v2