README.md

---
library_name: transformers
datasets:
- vector-institute/newsmediabias-plus
language:
- en
metrics:
- accuracy
- precision
- recall
- f1
base_model:
- google-bert/bert-base-uncased
pipeline_tag: text-classification
---

# BERT NMB+ (Disinformation Sequence Classification):

Classifies 512 chunks of a news article as "Likely" or "Unlikely" biased/disinformation.

Fine-tuned BERT ([bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased)) on the `headline`, `aritcle_text` and `text_label` fields in the [News Media Bias Plus Dataset](https://huggingface.co/datasets/vector-institute/newsmediabias-plus).

**This model was trained without weighted sampling, and the dataset contains 81.9% 'Likely' and 18.1% 'Unlikely' examples.** The same model trained with weighted sampling preformed worse on training eval metrics, but better when evaluated by gpt-4o-mini as a judge and is available [here](https://huggingface.co/maximuspowers/nmbp-bert-full-articles-balanced).

### Metics

*Evaluated on a 0.1 random sample of the NMB+ dataset, unseen during training*

- Accuracy:  0.7884
- Precision: 0.8573
- Recall:    0.8599
- F1 Score:  0.8586

## How to Use:

*Keep in mind, this model was trained on full 512 token chunks (tends to over-predict Unlikely for standalone sentences). If you're planning on processing stand alone sentences, you may find better results with this NMB+ model, which was trained on biased headlines.* 

```
from transformers import pipeline

classifier = pipeline("text-classification", model="maximuspowers/nmbp-bert-full-articles")
result = classifier("He was a terrible politician.", top_k=2)
```

### Example Response:
```
[
  {
    'label': 'Likely',
    'score': 0.9967995882034302
  },
  {
    'label': 'Unlikely',
    'score': 0.003200419945642352
  }
]
```