Benchmarking Bayesian Improved Surname Geocoding Against Machine Learning Methods
Bayesian Improved Surname Geocoding (BISG) is the most popular method for
proxying race/ethnicity in voter registration files that do not contain it.
This paper benchmarks BISG against a range of previously untested machine
learning alternatives, using voter files with self-reported race/ethnicity from
California, Florida, North Carolina, and Georgia. This analysis yields three
key findings. First, when given the exact same inputs, BISG and machine
learning perform similarly for estimating aggregate racial/ethnic composition.
Second, machine learning outperforms BISG at individual classification of
race/ethnicity. Third, the performance of all methods varies substantially
across states. These results suggest that pre-trained machine learning models
are preferable to BISG for individual classification. Furthermore, mixed
results at the precinct level and across states underscore the need for
researchers to empirically validate their chosen race/ethnicity proxy in their
populations of interest.