---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: polibert-malaysia-ver3
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# polibert-malaysia-ver3

This model is new version of YagiASAFAS/polibert-malaysia-ver2.
What is new is that this model used a new dataset which not only used tnwei/ms-newspapers dataset but also almost 10k of instagram posts regarding several topics about Malaysia.
By doing so, this model captures not only formal sentences such as News, but also captures informal sentences such as personal posts.
As a tradeoff, the accuracy was quite lower compared to the previous one
- Loss: 2.2001
- Accuracy: 0.6961

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 8
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 16
- mixed_precision_training: Native AMP

### Label Mappings
- 0: Economic Concerns
- 1: Racial discrimination or polarization
- 2: Leadership weaknesses
- 3: Development and infrastructure gaps
- 4: Corruption
- 5: Political instablility
- 6: Socials and Public safety
- 7: Administration
- 8: Education
- 9: Religion issues
- 10: Environmental

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 0.9899        | 1.0   | 2083  | 0.9351          | 0.7007   |
| 0.8717        | 2.0   | 4166  | 0.9117          | 0.7144   |
| 0.7277        | 3.0   | 6249  | 1.0107          | 0.7079   |
| 0.5764        | 4.0   | 8332  | 1.0944          | 0.7069   |
| 0.4703        | 5.0   | 10415 | 1.3644          | 0.7067   |
| 0.3377        | 6.0   | 12498 | 1.5634          | 0.7012   |
| 0.3188        | 7.0   | 14581 | 1.7475          | 0.6925   |
| 0.26          | 8.0   | 16664 | 1.7831          | 0.6944   |
| 0.206         | 9.0   | 18747 | 1.8693          | 0.6932   |
| 0.1994        | 10.0  | 20830 | 2.0837          | 0.6894   |
| 0.1718        | 11.0  | 22913 | 2.0601          | 0.6954   |
| 0.1635        | 12.0  | 24996 | 2.0624          | 0.6901   |
| 0.153         | 13.0  | 27079 | 2.0744          | 0.6863   |
| 0.1595        | 14.0  | 29162 | 2.1222          | 0.6925   |
| 0.153         | 15.0  | 31245 | 2.1712          | 0.6988   |
| 0.1188        | 16.0  | 33328 | 2.2001          | 0.6961   |


### Framework versions

- Transformers 4.18.0
- Pytorch 2.3.0+cu121
- Datasets 2.19.1
- Tokenizers 0.12.1