TITLE: Changes in European Solidarity Before and During COVID-19: Evidence from a Large Crowd- and Expert-Annotated Twitter Dataset 

ABSTRACT: We introduce the well-established social scientific concept of social
solidarity and its contestation, anti-solidarity, as a new problem setting to
supervised machine learning in NLP to assess how European solidarity discourses
changed before and after the COVID-19 outbreak was declared a global pandemic.
To this end, we annotate 2.3k English and German tweets for (anti-)solidarity
expressions, utilizing multiple human annotators and two annotation approaches
(experts vs.\ crowds). We use these annotations to train a BERT model with
multiple data augmentation strategies. Our augmented BERT model that combines
both expert and crowd annotations outperforms the baseline BERT classifier
trained with expert annotations only by over 25 points, from 58\% macro-F1 to
almost 85\%. We use this high-quality model to automatically label over 270k
tweets between September 2019 and December 2020. We then assess the
automatically labeled data for how statements related to European
(anti-)solidarity discourses developed over time and in relation to one
another, before and during the COVID-19 crisis. Our results show that
solidarity became increasingly salient and contested during the crisis. While
the number of solidarity tweets remained on a higher level and dominated the
discourse in the scrutinized time frame, anti-solidarity tweets initially
spiked, then decreased to (almost) pre-COVID-19 values before rising to a
stable higher level until the end of 2020.
We introduce the well-established social scientific concept of social
solidarity and its contestation, anti-solidarity, as a new problem setting to
supervised machine learning in NLP to assess how European solidarity discourses
changed before and after the COVID-19 outbreak was declared a global pandemic.
To this end, we annotate 2.3k English and German tweets for (anti-)solidarity
expressions, utilizing multiple human annotators and two annotation approaches
(experts vs.\ crowds). We use these annotations to train a BERT model with
multiple data augmentation strategies. Our augmented BERT model that combines
both expert and crowd annotations outperforms the baseline BERT classifier
trained with expert annotations only by over 25 points, from 58\% macro-F1 to
almost 85\%. We use this high-quality model to automatically label over 270k
tweets between September 2019 and December 2020. We then assess the
automatically labeled data for how statements related to European
(anti-)solidarity discourses developed over time and in relation to one
another, before and during the COVID-19 crisis. Our results show that
solidarity became increasingly salient and contested during the crisis. While
the number of solidarity tweets remained on a higher level and dominated the
discourse in the scrutinized time frame, anti-solidarity tweets initially
spiked, then decreased to (almost) pre-COVID-19 values before rising to a
stable higher level until the end of 2020.
Changes in European Solidarity Before and During COVID-19: Evidence from a Large Crowd- and Expert-Annotated Twitter Dataset
http://arxiv.org/abs/2108.01042v1