TITLE: A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication 

ABSTRACT: Datasets are central to training machine learning (ML) models. The ML
community has recently made significant improvements to data stewardship and
documentation practices across the model development life cycle. However, the
act of deprecating, or deleting, datasets has been largely overlooked, and
there are currently no standardized approaches for structuring this stage of
the dataset life cycle. In this paper, we study the practice of dataset
deprecation in ML, identify several cases of datasets that continued to
circulate despite having been deprecated, and describe the different technical,
legal, ethical, and organizational issues raised by such continuations. We then
propose a Dataset Deprecation Framework that includes considerations of risk,
mitigation of impact, appeal mechanisms, timeline, post-deprecation protocols,
and publication checks that can be adapted and implemented by the ML community.
Finally, we propose creating a centralized, sustainable repository system for
archiving datasets, tracking dataset modifications or deprecations, and
facilitating practices of care and stewardship that can be integrated into
research and publication processes.
Datasets are central to training machine learning (ML) models. The ML
community has recently made significant improvements to data stewardship and
documentation practices across the model development life cycle. However, the
act of deprecating, or deleting, datasets has been largely overlooked, and
there are currently no standardized approaches for structuring this stage of
the dataset life cycle. In this paper, we study the practice of dataset
deprecation in ML, identify several cases of datasets that continued to
circulate despite having been deprecated, and describe the different technical,
legal, ethical, and organizational issues raised by such continuations. We then
propose a Dataset Deprecation Framework that includes considerations of risk,
mitigation of impact, appeal mechanisms, timeline, post-deprecation protocols,
and publication checks that can be adapted and implemented by the ML community.
Finally, we propose creating a centralized, sustainable repository system for
archiving datasets, tracking dataset modifications or deprecations, and
facilitating practices of care and stewardship that can be integrated into
research and publication processes.
A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication
http://arxiv.org/abs/2111.04424v2