
# PARTYPRESS multilingual


Fine-tuned model in seven languages on texts from nine countries (Austria, Denmark, Germany, Ireland, Netherlands, Poland, Spain, Sweden, UK), based on [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased). Used in [Erfort et al. (2023)](https://doi.org/10.1177/20531680231183512), building on the PARTYPRESS database. For the downstream task of classyfing press releases from political parties into 23 unique policy areas we achieve a performance comparable to expert human coders.


## Model description

The PARTYPRESS multilingual model builds on [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) but has a supervised component. This means, it was fine-tuned using texts labeled by humans. The labels indicate 23 different political issue categories derived from the Comparative Agendas Project (CAP):

| Code | Issue |
|--|-------|
| 1 | Macroeconomics |
| 2 | Civil Rights |
| 3 | Health |
| 4 | Agriculture |
| 5 | Labor |
| 6 | Education |
| 7 | Environment |
| 8 | Energy |
| 9 | Immigration |
| 10 | Transportation |
| 12 | Law and Crime |
| 13 | Social Welfare |
| 14 | Housing |
| 15 | Domestic Commerce |
| 16 | Defense |
| 17 | Technology |
| 18 | Foreign Trade |
| 19.1 | International Affairs |
| 19.2 | European Union |
| 20 | Government Operations |
| 23 | Culture |
| 98 | Non-thematic |
| 99 | Other |

## Model variations

We also provide monolingual models for each of the nine countries covered by the PARTYPRESS database. The model can be easily extended to other languages, country contexts, or time periods by fine-tuning it with minimal additional labeled texts.

## Intended uses & limitations

The main use of the model is for text classification of press releases from political parties. It may also be useful for other political texts.

The classification can then be used to measure which issues parties are discussing in their communication.

### How to use

This model can be used directly with a pipeline for text classification:

```python
>>> from transformers import pipeline
>>> tokenizer_kwargs = {'padding':True,'truncation':True,'max_length':512}
>>> partypress = pipeline("text-classification", model = "cornelius/partypress-multilingual", tokenizer = "cornelius/partypress-multilingual", **tokenizer_kwargs)
>>> partypress(["We urgently need to fight climate change and reduce carbon emissions. This is what our party stands for.", 
            "We urge all parties to end the violence and come to the table. This conflict between the two countries must end.",
            "Así, “el trabajo de los militares españoles está al servicio de España y de los demás países”, que participan en esta misión por mandato de la OTAN, ha recordado.",
            "Dass es immer noch einen Gender-Pay-Gap gibt, geht auf das Konto dieser Regierung."])

[{'label': '7 - Environment', 'score': 0.9664431810379028},
 {'label': '19.1 - International Affairs', 'score': 0.9851641654968262},
 {'label': '16 - Defense', 'score': 0.986809492111206},
 {'label': '2 - Civil Rights', 'score': 0.9799079895019531}]


```

### Limitations and bias

The model was trained with data from parties in nine countries. For use in other countries, the model may be further fine-tuned. Without further fine-tuning, the performance of the model may be lower.

The model may have biased predictions. We discuss some biases by country, party, and over time in the release paper for the PARTYPRESS database. For example, the performance is highest for press releases from Ireland (75%) and lowest for Poland (55%).

## Training data

The PARTYPRESS multilingual model was fine-tuned with 27,243 press releases in seven languages on texts from 68 European parties in nine countries. The press releases were labeled by two expert human coders per country.

For the training data of the underlying model, please refer to [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased)

## Training procedure

### Preprocessing

For the preprocessing, please refer to [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased)

### Pretraining

For the pretraining, please refer to [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased)

### Fine-tuning

We fine-tuned the model using 27,243 labeled press releases from political parties in seven languages. 

#### Training Hyperparameters

The batch size for training was 12, for testing 2, with four epochs. All other hyperparameters were the standard from the transformers library.


#### Framework versions

- Transformers 4.28.0
- TensorFlow 2.12.0
- Datasets 2.12.0
- Tokenizers 0.13.3


## Evaluation results

Fine-tuned on our downstream task, this model achieves the following results in a five-fold cross validation that are comparable to the performance of our expert human coders:

| Accuracy | Precision | Recall  | F1 score |
|:--------:|:---------:|:-------:|:--------:|
|    69.52 |   67.99   | 67.60   |   66.77  |

Note that the classification task is difficult because topics such as environment and energy are often difficult to keep apart.

When we aggregate the shares of text for each issue, we find that the root-mean-square error is very low (0.29).

### BibTeX entry and citation info

```bibtex
@article{erfort_partypress_2023,
  author    = {Cornelius Erfort and
               Lukas F. Stoetzer and
               Heike Klüver},
  title     = {The PARTYPRESS Database: A new comparative database of parties’ press releases},
  journal   = {Research and Politics},
  volume    = {10},
  number    = {3},
  year      = {2023},
  doi       = {10.1177/20531680231183512},
  URL       = {https://doi.org/10.1177/20531680231183512}

}
```

Erfort, C., Stoetzer, L. F., & Klüver, H. (2023). The PARTYPRESS Database: A new comparative database of parties’ press releases. Research & Politics, 10(3). [https://doi.org/10.1177/20531680231183512](https://doi.org/10.1177/20531680231183512)

### Further resources

Github: [cornelius-erfort/partypress](https://github.com/cornelius-erfort/partypress)

Research and Politics Dataverse: [Replication Data for: The PARTYPRESS Database: A New Comparative Database of Parties’ Press Releases](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi%3A10.7910%2FDVN%2FOINX7Q)



## Acknowledgements

Research for this contribution is part of the Cluster of Excellence "Contestations of the Liberal Script" (EXC 2055, Project-ID: 390715649), funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy. Cornelius Erfort is moreover grateful for generous funding provided by the DFG through the Research Training Group DYNAMICS (GRK 2458/1).

## Contact

Cornelius Erfort

Humboldt-Universität zu Berlin

[corneliuserfort.de](corneliuserfort.de)



{"language": ["de", "en", "es", "da", "pl", "sv", "nl"], "license": "cc-by-sa-4.0", "tags": ["partypress", "political science", "parties", "press releases"], "metrics": ["accuracy"], "pipeline_tag": "text-classification", "widget": [{"text": "Labour launches air pollution campaign Labour\u2019s Shadow Minister for the Natural Environment, will today (Wednesday) launch Labour\u2019s campaign against air pollution. Maria Eagle will say that 29,000 people die prematurely in the UK each year because of poor air pollution in our towns and cities - including more than 3,000 in London. Scientists have warned that air pollution in Britain\u2019s most polluted cities is stunting the development of children\u2019s lungs. Maria Eagle MP and Sadiq Khan MP will announce that the next Labour Government will deliver a national framework for Low Emission Zones to enable local authorities to encourage cleaner, greener, less-polluting vehicles to begin to tackle this problem. Unlike this Tory-led Government the Labour Party will devolve the power, not just the responsibility, to Local Authorities willing take action against air pollution."}, {"text": "Dawn Butler MP, Labour\u2019s Shadow\nWomen and Equalities Secretary, commenting on Equal Pay Day today, said: \u201cFrom today onwards women effectively work the rest of the year for free, which means fifty days of unpaid labour until we hit 2018. \u201cThe fact that the gender pay gap has remained the same for the past three years is a shocking indictment of the Government\u2019s failure to tackle unequal pay and the underlying structural issues that allow these disparities to exist. \u201cLabour is the party of equality. It was Labour that introduced the Equal Pay Act in 1970 and the next Labour Government will take the necessary action to end the scourge of unequal pay once and for all."}, {"text": "\"Los soldados espa\u00f1oles que est\u00e1n en Afganist\u00e1n cuentan con las m\u00e1ximas medidas de seguridad para su protecci\u00f3n\" Jes\u00fas Cuadrado, portavoz socialista en la Comisi\u00f3n de Defensa del Congreso, ha enviado esta tarde, en nombre del PSOE, un mensaje de apoyo y solidaridad a los soldados heridos hoy en Afganist\u00e1n y a sus familias, as\u00ed como a sus compa\u00f1eros en esta misi\u00f3n internacional. Cuadrado ha resaltado el \u201cenorme sacrificio\u201d que supone para los soldados una misi\u00f3n de estas caracter\u00edsticas. Un sacrificio \u201cque est\u00e1 al servicio de todo los espa\u00f1oles\u201d, ha explicado, \u201cporque contribuyen a la creaci\u00f3n de un Estado\u201d, en un lugar que ha sido usado hasta el momento por los terroristas para cometer atentados en su pa\u00eds y en resto del mundo.  Gracias al trabajo de \u201cnuestros soldados\u201d, ha a\u00f1adido, \u201cahora hay un ej\u00e9rcito compuesto por militares afganos y hay una polic\u00eda\u201d. As\u00ed, \u201cel trabajo de los militares espa\u00f1oles est\u00e1 al servicio de Espa\u00f1a y de los dem\u00e1s pa\u00edses\u201d, que participan en esta misi\u00f3n por mandato de la OTAN, ha recordado. \u201cEs uno de los trabajos m\u00e1s solidarios y comprometidos que se pueden hacer en el mundo\u201d, ha resaltado el portavoz socialista.  La seguridad de nuestras tropas, una prioridad absoluta  \u201cLa seguridad al cien por cien es imposible\u201d, ha admitido Cuadrado. As\u00ed lo ha se\u00f1alado en numerosas ocasiones tanto el Ministerio de Defensa como el resto del Gobierno. \u201cLos riesgos que asumimos all\u00ed son muy elevados y, precisamente por eso, durante estos a\u00f1os, el Gobierno ha hecho de la seguridad de nuestras tropas una prioridad absoluta\u201d.  Cuadrado ha recordado que \u201ctodos los blindados que utilizan los militares espa\u00f1oles en Afganist\u00e1n han sido renovados\u201d. Los soldados espa\u00f1oles, en concreto, cuentan con 67 RG31 y 131 blindados tipo Lince. Tambi\u00e9n se ha construido una nueva base y se ha dotado, a todo el material que utilizan, de las m\u00e1s modernas medidas de seguridad, as\u00ed como de unos servicios sanitarios de alto nivel. \u201cIgualmente, es conocido que se ha ido mejorando el sistema de transporte de nuestras tropas con las mejores medidas de seguridad\u201d, concluy\u00f3 el portavoz en la Comisi\u00f3n de Defensa."}, {"text": "Nederland kan uit de crisis komen als we banen en structureel herstel voorop stellen. Dat zei Sybrand Buma vandaag in zijn speech op het partijcongres van het CDA in Leeuwarden. Dat was volgens hem de aanpak van Ruud Lubbers in de crisis van de jaren 80, en het was de aanpak van Jan Peter Balkenende aan het begin van deze eeuw. Beide CDA\u2019ers waren er van overtuigd dat werk en economisch herstel niet worden gecre\u00eberd door de overheid. Maar de overheid kan wel de omstandigheden scheppen waaronder de economie weer tot leven komt. Die visie van Lubbers en Balkenende is ook de visie van Buma en ligt ten grondslag aan het alternatief dat hij begin september presenteerde.  Buma benadrukte ook dat het CDA zowel het kabinet als D66, CU en SGP zal blijven aansporen om de chronisch zieken en gehandicapten meer tegemoet te komen. Het herfstakkoord is in strijd met alles waar het CDA vanaf de jaren \u201980, tot en met het Strategisch Beraad, voor staat. Het CDA wil een bloeiende samenleving met een eerlijke economie en een dienende overheid als schild voor de zwakken.  Buma stond er ook nog even bij stil dat Leeuwarden voor hem een bijzondere plek is, om twee redenen. Hij is er geboren, en het is de plaats van waaruit de Elfstedentocht begint. Buma heeft zelf de Elfstedentocht gereden, bijna 30 jaar geleden, en is nog steeds rijdend lid van de vereniging. Tijdens zijn speech maakte hij bekend dat hij zijn startbewijs inleverde om ruimte te maken voor een jongere, mocht de tocht binnenkort weer gereden worden.Klik op de link op onderaan deze pagina om de hele speech te lezen."}]}
transformers pytorch tf safetensors bert text-classification partypress political science parties press releases de en es da pl sv nl license:cc-by-sa-4.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #tf #safetensors #bert #text-classification #partypress #political science #parties #press releases #de #en #es #da #pl #sv #nl #license-cc-by-sa-4.0 #autotrain_compatible #endpoints_compatible #region-us 

PARTYPRESS multilingual
=======================


Fine-tuned model in seven languages on texts from nine countries (Austria, Denmark, Germany, Ireland, Netherlands, Poland, Spain, Sweden, UK), based on bert-base-multilingual-cased. Used in Erfort et al. (2023), building on the PARTYPRESS database. For the downstream task of classyfing press releases from political parties into 23 unique policy areas we achieve a performance comparable to expert human coders.


Model description
-----------------


The PARTYPRESS multilingual model builds on bert-base-multilingual-cased but has a supervised component. This means, it was fine-tuned using texts labeled by humans. The labels indicate 23 different political issue categories derived from the Comparative Agendas Project (CAP):



Model variations
----------------


We also provide monolingual models for each of the nine countries covered by the PARTYPRESS database. The model can be easily extended to other languages, country contexts, or time periods by fine-tuning it with minimal additional labeled texts.


Intended uses & limitations
---------------------------


The main use of the model is for text classification of press releases from political parties. It may also be useful for other political texts.


The classification can then be used to measure which issues parties are discussing in their communication.


### How to use


This model can be used directly with a pipeline for text classification:


### Limitations and bias


The model was trained with data from parties in nine countries. For use in other countries, the model may be further fine-tuned. Without further fine-tuning, the performance of the model may be lower.


The model may have biased predictions. We discuss some biases by country, party, and over time in the release paper for the PARTYPRESS database. For example, the performance is highest for press releases from Ireland (75%) and lowest for Poland (55%).


Training data
-------------


The PARTYPRESS multilingual model was fine-tuned with 27,243 press releases in seven languages on texts from 68 European parties in nine countries. The press releases were labeled by two expert human coders per country.


For the training data of the underlying model, please refer to bert-base-multilingual-cased


Training procedure
------------------


### Preprocessing


For the preprocessing, please refer to bert-base-multilingual-cased


### Pretraining


For the pretraining, please refer to bert-base-multilingual-cased


### Fine-tuning


We fine-tuned the model using 27,243 labeled press releases from political parties in seven languages.


#### Training Hyperparameters


The batch size for training was 12, for testing 2, with four epochs. All other hyperparameters were the standard from the transformers library.


#### Framework versions


* Transformers 4.28.0
* TensorFlow 2.12.0
* Datasets 2.12.0
* Tokenizers 0.13.3


Evaluation results
------------------


Fine-tuned on our downstream task, this model achieves the following results in a five-fold cross validation that are comparable to the performance of our expert human coders:



Note that the classification task is difficult because topics such as environment and energy are often difficult to keep apart.


When we aggregate the shares of text for each issue, we find that the root-mean-square error is very low (0.29).


### BibTeX entry and citation info


Erfort, C., Stoetzer, L. F., & Klüver, H. (2023). The PARTYPRESS Database: A new comparative database of parties’ press releases. Research & Politics, 10(3). URL


### Further resources


Github: cornelius-erfort/partypress


Research and Politics Dataverse: Replication Data for: The PARTYPRESS Database: A New Comparative Database of Parties’ Press Releases


Acknowledgements
----------------


Research for this contribution is part of the Cluster of Excellence "Contestations of the Liberal Script" (EXC 2055, Project-ID: 390715649), funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy. Cornelius Erfort is moreover grateful for generous funding provided by the DFG through the Research Training Group DYNAMICS (GRK 2458/1).


Contact
-------


Cornelius Erfort


Humboldt-Universität zu Berlin


URL



### How to use


This model can be used directly with a pipeline for text classification: ### Limitations and bias


The model was trained with data from parties in nine countries. For use in other countries, the model may be further fine-tuned. Without further fine-tuning, the performance of the model may be lower.


The model may have biased predictions. We discuss some biases by country, party, and over time in the release paper for the PARTYPRESS database. For example, the performance is highest for press releases from Ireland (75%) and lowest for Poland (55%).


Training data
-------------


The PARTYPRESS multilingual model was fine-tuned with 27,243 press releases in seven languages on texts from 68 European parties in nine countries. The press releases were labeled by two expert human coders per country.


For the training data of the underlying model, please refer to bert-base-multilingual-cased


Training procedure
------------------ ### Preprocessing


For the preprocessing, please refer to bert-base-multilingual-cased ### Pretraining


For the pretraining, please refer to bert-base-multilingual-cased ### Fine-tuning


We fine-tuned the model using 27,243 labeled press releases from political parties in seven languages. #### Training Hyperparameters


The batch size for training was 12, for testing 2, with four epochs. All other hyperparameters were the standard from the transformers library. #### Framework versions


* Transformers 4.28.0
* TensorFlow 2.12.0
* Datasets 2.12.0
* Tokenizers 0.13.3


Evaluation results
------------------


Fine-tuned on our downstream task, this model achieves the following results in a five-fold cross validation that are comparable to the performance of our expert human coders:



Note that the classification task is difficult because topics such as environment and energy are often difficult to keep apart.


When we aggregate the shares of text for each issue, we find that the root-mean-square error is very low (0.29). ### BibTeX entry and citation info


Erfort, C., Stoetzer, L. F., & Klüver, H. (2023). The PARTYPRESS Database: A new comparative database of parties’ press releases. Research & Politics, 10(3). URL ### Further resources


Github: cornelius-erfort/partypress


Research and Politics Dataverse: Replication Data for: The PARTYPRESS Database: A New Comparative Database of Parties’ Press Releases


Acknowledgements
----------------


Research for this contribution is part of the Cluster of Excellence "Contestations of the Liberal Script" (EXC 2055, Project-ID: 390715649), funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy. Cornelius Erfort is moreover grateful for generous funding provided by the DFG through the Research Training Group DYNAMICS (GRK 2458/1).


Contact
-------


Cornelius Erfort


Humboldt-Universität zu Berlin


URL
TAGS
#transformers #pytorch #tf #safetensors #bert #text-classification #partypress #political science #parties #press releases #de #en #es #da #pl #sv #nl #license-cc-by-sa-4.0 #autotrain_compatible #endpoints_compatible #region-us 
 ### How to use


This model can be used directly with a pipeline for text classification: ### Limitations and bias


The model was trained with data from parties in nine countries. For use in other countries, the model may be further fine-tuned. Without further fine-tuning, the performance of the model may be lower.


The model may have biased predictions. We discuss some biases by country, party, and over time in the release paper for the PARTYPRESS database. For example, the performance is highest for press releases from Ireland (75%) and lowest for Poland (55%).


Training data
-------------


The PARTYPRESS multilingual model was fine-tuned with 27,243 press releases in seven languages on texts from 68 European parties in nine countries. The press releases were labeled by two expert human coders per country.


For the training data of the underlying model, please refer to bert-base-multilingual-cased


Training procedure
------------------ ### Preprocessing


For the preprocessing, please refer to bert-base-multilingual-cased ### Pretraining


For the pretraining, please refer to bert-base-multilingual-cased ### Fine-tuning


We fine-tuned the model using 27,243 labeled press releases from political parties in seven languages. #### Training Hyperparameters


The batch size for training was 12, for testing 2, with four epochs. All other hyperparameters were the standard from the transformers library. #### Framework versions


* Transformers 4.28.0
* TensorFlow 2.12.0
* Datasets 2.12.0
* Tokenizers 0.13.3


Evaluation results
------------------


Fine-tuned on our downstream task, this model achieves the following results in a five-fold cross validation that are comparable to the performance of our expert human coders:



Note that the classification task is difficult because topics such as environment and energy are often difficult to keep apart.


When we aggregate the shares of text for each issue, we find that the root-mean-square error is very low (0.29). ### BibTeX entry and citation info


Erfort, C., Stoetzer, L. F., & Klüver, H. (2023). The PARTYPRESS Database: A new comparative database of parties’ press releases. Research & Politics, 10(3). URL ### Further resources


Github: cornelius-erfort/partypress


Research and Politics Dataverse: Replication Data for: The PARTYPRESS Database: A New Comparative Database of Parties’ Press Releases


Acknowledgements
----------------


Research for this contribution is part of the Cluster of Excellence "Contestations of the Liberal Script" (EXC 2055, Project-ID: 390715649), funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy. Cornelius Erfort is moreover grateful for generous funding provided by the DFG through the Research Training Group DYNAMICS (GRK 2458/1).


Contact
-------


Cornelius Erfort


Humboldt-Universität zu Berlin


URL