---
license: apache-2.0
base_model: Twitter/twhin-bert-base
tags:
- generated_from_trainer
metrics:
- f1
model-index:
- name: twhin-bert-base
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# twhin-bert-base

This model is a fine-tuned version of [Twitter/twhin-bert-base](https://huggingface.co/Twitter/twhin-bert-base) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6341
- F1: 0.3077

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-07
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step | Validation Loss | F1     |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 0.6867        | 1.0   | 189  | 0.6817          | 0.1026 |
| 0.684         | 2.0   | 378  | 0.6746          | 0.0571 |
| 0.675         | 3.0   | 567  | 0.6649          | 0.0    |
| 0.6642        | 4.0   | 756  | 0.6577          | 0.0    |
| 0.6653        | 5.0   | 945  | 0.6542          | 0.0    |
| 0.6649        | 6.0   | 1134 | 0.6479          | 0.0    |
| 0.6648        | 7.0   | 1323 | 0.6460          | 0.0    |
| 0.6511        | 8.0   | 1512 | 0.6387          | 0.0    |
| 0.6535        | 9.0   | 1701 | 0.6332          | 0.0    |
| 0.6544        | 10.0  | 1890 | 0.6261          | 0.0    |
| 0.6382        | 11.0  | 2079 | 0.6154          | 0.0    |
| 0.6315        | 12.0  | 2268 | 0.6051          | 0.0    |
| 0.6231        | 13.0  | 2457 | 0.5957          | 0.2326 |
| 0.603         | 14.0  | 2646 | 0.5858          | 0.2326 |
| 0.6034        | 15.0  | 2835 | 0.5771          | 0.2553 |
| 0.5938        | 16.0  | 3024 | 0.5694          | 0.2308 |
| 0.5884        | 17.0  | 3213 | 0.5642          | 0.3103 |
| 0.5763        | 18.0  | 3402 | 0.5611          | 0.3103 |
| 0.5675        | 19.0  | 3591 | 0.5641          | 0.2857 |
| 0.5672        | 20.0  | 3780 | 0.5598          | 0.3000 |
| 0.5674        | 21.0  | 3969 | 0.5579          | 0.2857 |
| 0.5479        | 22.0  | 4158 | 0.5642          | 0.3125 |
| 0.5621        | 23.0  | 4347 | 0.5688          | 0.2903 |
| 0.5516        | 24.0  | 4536 | 0.5685          | 0.3077 |
| 0.5597        | 25.0  | 4725 | 0.5713          | 0.3077 |
| 0.5418        | 26.0  | 4914 | 0.5761          | 0.3077 |
| 0.5477        | 27.0  | 5103 | 0.5752          | 0.3030 |
| 0.535         | 28.0  | 5292 | 0.5876          | 0.3077 |
| 0.5544        | 29.0  | 5481 | 0.5841          | 0.3030 |
| 0.5238        | 30.0  | 5670 | 0.5855          | 0.3030 |
| 0.5375        | 31.0  | 5859 | 0.5894          | 0.3030 |
| 0.5092        | 32.0  | 6048 | 0.5985          | 0.3077 |
| 0.5262        | 33.0  | 6237 | 0.5988          | 0.3077 |
| 0.5418        | 34.0  | 6426 | 0.6038          | 0.3077 |
| 0.531         | 35.0  | 6615 | 0.6087          | 0.3077 |
| 0.5627        | 36.0  | 6804 | 0.6064          | 0.3077 |
| 0.545         | 37.0  | 6993 | 0.6110          | 0.3077 |
| 0.5105        | 38.0  | 7182 | 0.6134          | 0.3077 |
| 0.5471        | 39.0  | 7371 | 0.6111          | 0.3077 |
| 0.5114        | 40.0  | 7560 | 0.6212          | 0.3077 |
| 0.5411        | 41.0  | 7749 | 0.6159          | 0.3077 |
| 0.5304        | 42.0  | 7938 | 0.6213          | 0.3077 |
| 0.5146        | 43.0  | 8127 | 0.6276          | 0.3077 |
| 0.5223        | 44.0  | 8316 | 0.6301          | 0.3077 |
| 0.5345        | 45.0  | 8505 | 0.6281          | 0.3077 |
| 0.5368        | 46.0  | 8694 | 0.6284          | 0.3077 |
| 0.516         | 47.0  | 8883 | 0.6320          | 0.3077 |
| 0.5241        | 48.0  | 9072 | 0.6339          | 0.3077 |
| 0.5267        | 49.0  | 9261 | 0.6342          | 0.3077 |
| 0.5478        | 50.0  | 9450 | 0.6341          | 0.3077 |


### Framework versions

- Transformers 4.32.1
- Pytorch 2.1.2
- Datasets 2.12.0
- Tokenizers 0.13.3