TITLE: A Dataset and an Approach for Identity Resolution of 38 Million Author IDs extracted from 2B Git Commits 

ABSTRACT: The data collected from open source projects provide means to model large
software ecosystems, but often suffer from data quality issues, specifically,
multiple author identification strings in code commits might actually be
associated with one developer. While many methods have been proposed for
addressing this problem, they are either heuristics requiring manual tweaking,
or require too much calculation time to do pairwise comparisons for 38M author
IDs in, for example, the World of Code collection. In this paper, we propose a
method that finds all author IDs belonging to a single developer in this entire
dataset, and share the list of all author IDs that were found to have aliases.
To do this, we first create blocks of potentially connected author IDs and then
use a machine learning model to predict which of these potentially related IDs
belong to the same developer. We processed around 38 million author IDs and
found around 14.8 million IDs to have an alias, which belong to 5.4 million
different developers, with the median number of aliases being 2 per developer.
This dataset can be used to create more accurate models of developer behaviour
at the entire OSS ecosystem level and can be used to provide a service to
rapidly resolve new author IDs.
The data collected from open source projects provide means to model large
software ecosystems, but often suffer from data quality issues, specifically,
multiple author identification strings in code commits might actually be
associated with one developer. While many methods have been proposed for
addressing this problem, they are either heuristics requiring manual tweaking,
or require too much calculation time to do pairwise comparisons for 38M author
IDs in, for example, the World of Code collection. In this paper, we propose a
method that finds all author IDs belonging to a single developer in this entire
dataset, and share the list of all author IDs that were found to have aliases.
To do this, we first create blocks of potentially connected author IDs and then
use a machine learning model to predict which of these potentially related IDs
belong to the same developer. We processed around 38 million author IDs and
found around 14.8 million IDs to have an alias, which belong to 5.4 million
different developers, with the median number of aliases being 2 per developer.
This dataset can be used to create more accurate models of developer behaviour
at the entire OSS ecosystem level and can be used to provide a service to
rapidly resolve new author IDs.
A Dataset and an Approach for Identity Resolution of 38 Million Author IDs extracted from 2B Git Commits
http://arxiv.org/abs/2003.08349v2