task_categories:text-classification task_ids:sentiment-classification annotations_creators:mixed multilinguality:multi-lingual size_categories:1M<n<10M language:ar language:bg language:bs language:cs language:de language:el language:en language:es language:fa language:fr language:he language:hi language:hr language:hu language:it language:ja language:lv language:pl language:pt language:ru language:sk language:sl language:sq language:sr language:sv language:th language:ur language:zh license:other arxiv:2306.07902 region:us
    This work presents the most extensive open massively multi-lingual corpus of datasets for training sentiment models. 
    The corpus consists of 79 manually selected from over 350 datasets reported in the scientific literature based on strict quality criteria and covers 25 languages. 
    Datasets can be queried using several linguistic and functional features. 
    In addition, we present a multi-faceted sentiment classification benchmark summarizing hundreds of experiments conducted on different base models, training objectives, dataset collections, and fine-tuning strategies.
@misc{augustyniak2023massively,
      title={Massively Multilingual Corpus of Sentiment Datasets and Multi-faceted Sentiment Classification Benchmark}, 
      author={Łukasz Augustyniak and Szymon Woźniak and Marcin Gruza and Piotr Gramacki and Krzysztof Rajda and Mikołaj Morzy and Tomasz Kajdanowicz},
      year={2023},
      eprint={2306.07902},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
---
annotations_creators:
- mixed
language:
- ar
- bg
- bs
- cs
- de
- el
- en
- es
- fa
- fr
- he
- hi
- hr
- hu
- it
- ja
- lv
- pl
- pt
- ru
- sk
- sl
- sq
- sr
- sv
- th
- ur
- zh
license:
- other
multilinguality:
- multi-lingual
size_categories:
- 1M<n<10M
task_categories:
- text-classification
task_ids:
- sentiment-classification
pretty_name: Massive-Multilingual-Sentiment
---

# Massive Multilingual Sentiment Corpora (MMS)

## Corpora Summary

Despite impressive advancements in multilingual corpora collection and model training, developing large-scale deployments of multilingual models still presents a significant challenge. This is particularly true for language tasks that are culture-dependent. One such example is the area of multilingual sentiment analysis, where affective markers can be subtle and deeply ensconced in culture.

This work presents the most extensive open massively multilingual corpus of datasets for training sentiment models. The corpus consists of 79 manually selected from over 350 datasets reported in the scientific literature based on strict quality criteria and covers 27 languages. Datasets can be queried using several linguistic and functional features. In addition, we present a multi-faceted sentiment classification benchmark summarizing hundreds of experiments conducted on different base models, training objectives, dataset collections, and fine-tuning strategies. 

More about dataset here [https://brand24-ai.github.io/mms_benchmark](https://brand24-ai.github.io/mms_benchmark).

## General licenses information

This is a library of the open-sourced datasets that we gathered. We provide citations or links to sources of these datasets. It is essential to mention that these datasets could have different licenses, and we encourage everybody to check the permissions of each dataset separately. It is critical because, for example, not all datasets will be available for commercial purposes. This ensures that proper consent and permissions are obtained for the use and curation of the data, respecting the rights and privacy of the individuals whose data is included in the datasets. You will cite our library and the authors of each dataset you want to use. 

## Usage

```python
import datasets

# whole dataset will be downloaded and cached
mms_dataset = datasets.load_dataset("Brand24/mms")

# filter only texts in Polish
pl = mms_dataset.filter(lambda row: row['language'] == 'pl')
```
## Corpora statistics

### Per language

| language   | label_name   |   count |
|:-----------|:-------------|--------:|
| ar         | negative     |  138899 |
| ar         | neutral      |  192774 |
| ar         | positive     |  600402 |
| bg         | negative     |   13930 |
| bg         | neutral      |   28657 |
| bg         | positive     |   19563 |
| bs         | negative     |   11974 |
| bs         | neutral      |   11145 |
| bs         | positive     |   13064 |
| cs         | negative     |   39674 |
| cs         | neutral      |   59200 |
| cs         | positive     |   97413 |
| de         | negative     |  104667 |
| de         | neutral      |  100071 |
| de         | positive     |  111149 |
| el         | negative     |     230 |
| el         | neutral      |      38 |
| el         | positive     |     232 |
| en         | negative     |  304939 |
| en         | neutral      |  290823 |
| en         | positive     | 1734724 |
| es         | negative     |  108733 |
| es         | neutral      |  122493 |
| es         | positive     |  187486 |
| fa         | negative     |    1602 |
| fa         | neutral      |    5091 |
| fa         | positive     |    6832 |
| fr         | negative     |   84187 |
| fr         | neutral      |   43245 |
| fr         | positive     |   83199 |
| he         | negative     |    2279 |
| he         | neutral      |     243 |
| he         | positive     |    6097 |
| hi         | negative     |    4992 |
| hi         | neutral      |    6392 |
| hi         | positive     |    5615 |
| hr         | negative     |   19757 |
| hr         | neutral      |   19470 |
| hr         | positive     |   38367 |
| hu         | negative     |    8974 |
| hu         | neutral      |   17621 |
| hu         | positive     |   30087 |
| it         | negative     |    4043 |
| it         | neutral      |    4193 |
| it         | positive     |    3829 |
| ja         | negative     |   83982 |
| ja         | neutral      |   41979 |
| ja         | positive     |   83819 |
| lv         | negative     |    1378 |
| lv         | neutral      |    2618 |
| lv         | positive     |    1794 |
| pl         | negative     |   77422 |
| pl         | neutral      |   62074 |
| pl         | positive     |   97192 |
| pt         | negative     |   56827 |
| pt         | neutral      |   55165 |
| pt         | positive     |   45842 |
| ru         | negative     |   31770 |
| ru         | neutral      |   48106 |
| ru         | positive     |   31054 |
| sk         | negative     |   14431 |
| sk         | neutral      |   12842 |
| sk         | positive     |   29350 |
| sl         | negative     |   33694 |
| sl         | neutral      |   50553 |
| sl         | positive     |   29296 |
| sq         | negative     |    6889 |
| sq         | neutral      |   14757 |
| sq         | positive     |   22638 |
| sr         | negative     |   25089 |
| sr         | neutral      |   32283 |
| sr         | positive     |   18996 |
| sv         | negative     |   16266 |
| sv         | neutral      |   13342 |
| sv         | positive     |   11738 |
| th         | negative     |    9326 |
| th         | neutral      |   28616 |
| th         | positive     |   34377 |
| ur         | negative     |    5239 |
| ur         | neutral      |    8585 |
| ur         | positive     |    5836 |
| zh         | negative     |  117967 |
| zh         | neutral      |   69016 |
| zh         | positive     |  144719 |

## Dataset Structure

### Linguistic Typology

The field of language typology focuses on studying the similarities and differences among languages. These differences can be categorized into phonological (sounds), syntactic (structures), lexical (vocabulary), and theoretical aspects. Linguistic typology analyzes the current state of languages, contrasting with genealogical linguistics, which examines historical relationships between languages.

Genealogical linguistics studies language families and genera. A language family consists of languages that share a common ancestral language, while genera are branches within a language family. The Indo-European family, for example, includes genera such as Slavic, Romance, Germanic, and Indic. Over 7000 languages are categorized into approximately 150 language families, with Indo-European, Sino-Tibetan, Turkic, Afro-Asiatic, Nilo-Saharan, Niger-Congo, and Eskimo-Aleut being some of the largest families.

Within linguistic typology, languages are described using various linguistic features. Our work focuses on sentiment classification and selects ten relevant features:

- `text`: The feature text represents the actual text of the sentiment dataset. It is of type string and contains the text samples or sentences for sentiment analysis.
- `label`: The feature label corresponds to the sentiment labels of the text samples. It is of type ClassLabel and has three possible values: negative, neutral, and positive. These labels indicate the sentiment or emotional polarity associated with the text.
- `original_dataset`: The feature original_dataset refers to the name or identifier of the original dataset from which the text samples were extracted. It is of type string and provides information about the source dataset.
- `domain`: The feature domain represents the domain or topic of the sentiment dataset. It is of type string and provides context regarding the subject matter of the text samples.
- `language`: The feature language indicates the language of the text samples in the sentiment dataset. It is of type string and specifies the language in which the text is written.
- `Family`: The feature Family represents the language family to which a specific language belongs. It is of type string and provides information about the broader categorization of languages into language families.
- `Genus`: The feature Genus corresponds to the genus or branch within a language family. It is of type string and indicates the specific subgrouping of languages within a language family.
- `Definite article`: Half of the languages do not use the definite article, which signals uniqueness or definiteness of a concept.
- `Indefinite article`: Half of the languages do not use the indefinite article, with some languages using a separate article or the numeral "one."
- `Number of cases`: Languages vary greatly in the number of morphological cases used.
- `Order of subject, verb, and object`: Different languages have different word orderings, with variations like SOV, SVO, VSO, VOS, OVS, and OSV.
- `Negative morphemes`: Negative morphemes indicate clausal negation in declarative sentences.
- `Polar questions`: Questions with yes/no answers, which can be formed using question particles, interrogative morphology, or intonation.
- `Position of the negative morpheme`: The position of the negative morpheme can vary in relation to subjects and objects.
- `Prefixing vs. suffixing`: Languages differ in their use of prefixes and suffixes in inflectional morphology.
- `Coding of nominal plurals`: Plurals can be expressed through morphological changes or the use of plurality indicator morphemes.
- `Grammatical genders`: Languages vary in the number of grammatical genders used, or may not use the concept at all.

These language features are available as filtering options in our library. Users can download specific facets of the collection, such as datasets in Slavic languages with interrogative word order for polar questions or datasets from the Afro-Asiatic language family without morphological case-making.

### Usage 

Code example for loading and filtering Slavic language in which polar questions are formed using the interrogative word order

```python
import datasets

mms_dataset = datasets.load_dataset("Brand24/mms")
slavic = mms_dataset.filter(lambda row: row["Genus"] == "Slavic" and row["Polar questions"] == "interrogative word order")
```

Filtering sentiment datasets from the Afro-Asiatic language family without morphological case-making

```python
afro_asiatic = mms_dataset.filter(lambda row: row["Family"] == "Afro-Asiatic" and row["Number of cases"] == "no morphological case-making")
```

## Dataset Creation

### Who are the source language producers?

The data comes from multiple papers and covers a large variety of languages. For the specific dataset information, please check out the companion paper.

### Annotations

Similarly, like for data producers, you should check papers that propose the specific datasets you are interested in. 

#### Annotation process

We describe the annotations process of our internally created dataset in this corpus.

## Considerations for Using the Data

### Social Impact and Limitations

Corpus is intended to bring more sentiment annotated data to a wide variety of lanuages, the aim of the corpus is to make large amounts of data available to lower resource languages in order to facilitate the training of state-of-the-art ML models for sentiment analysis.

## Additional Information

### Dataset Curators

The corpus was put together by

- [@laugustyniak](https://www.linkedin.com/in/lukaszaugustyniak/)
- [@swozniak](https://www.linkedin.com/in/wscode/)
- [@mgruza](https://www.linkedin.com/in/marcin-gruza-276b2512b/)
- [@pgramacki](https://www.linkedin.com/in/piotrgramacki/)
- [@krajda](https://www.linkedin.com/in/krzysztof-rajda/)
- [@mmorzy](https://www.linkedin.com/in/mikolajmorzy/)
- [@tkajdanowicz](https://www.linkedin.com/in/kajdanowicz/)

### Licensing Information

These data are released under this licensing scheme.
We do not own any text from which these data and datasets have been extracted.

We license the actual packaging of these data under the Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) https://creativecommons.org/licenses/by-nc/4.0/

This work is published from Poland.

Should you consider that our data contains material that is owned by you and should, therefore not be reproduced here, please:
* Clearly identify yourself with detailed contact data such as an address, telephone number, or email address at which you can be contacted.
* Clearly identify the copyrighted work claimed to be infringed.
* Clearly identify the material claimed to be infringing and the information reasonably sufficient to allow us to locate the material.

We will comply with legitimate requests by removing the affected sources from the next release of the corpus.


### Citation Information

### The main corpus citation

```bibtex
@misc{augustyniak2023massively,
      title={Massively Multilingual Corpus of Sentiment Datasets and Multi-faceted Sentiment Classification Benchmark}, 
      author={Łukasz Augustyniak and Szymon Woźniak and Marcin Gruza and Piotr Gramacki and Krzysztof Rajda and Mikołaj Morzy and Tomasz Kajdanowicz},
      year={2023},
      eprint={2306.07902},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```

### All datasets in corpus

[https://brand24-ai.github.io/mms_benchmark/citations.html](https://brand24-ai.github.io/mms_benchmark/citations.html)

## Acknowledgements

- BRAND24 - https://brand24.com
- CLARIN-PL-Biz - https://clarin.biz
