
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# legal_bert_sm_gen1_large

This model is a fine-tuned version of [nlpaueb/legal-bert-base-uncased](https://huggingface.co/nlpaueb/legal-bert-base-uncased) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.3342
- Accuracy: 0.8342
- Precision: 0.6462
- Recall: 0.3993
- F1: 0.4936
- D-index: 1.6166

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 96000
- num_epochs: 20
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy | Precision | Recall | F1     | D-index |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:---------:|:------:|:------:|:-------:|
| 0.4153        | 1.0   | 1500  | 0.3934          | 0.8279   | 0.7013    | 0.2610 | 0.3804 | 1.5630  |
| 0.3682        | 2.0   | 3000  | 0.3595          | 0.8448   | 0.6802    | 0.4405 | 0.5347 | 1.6439  |
| 0.3509        | 3.0   | 4500  | 0.3559          | 0.847    | 0.7258    | 0.3923 | 0.5094 | 1.6314  |
| 0.3266        | 4.0   | 6000  | 0.3545          | 0.8484   | 0.7335    | 0.3944 | 0.5130 | 1.6339  |
| 0.2927        | 5.0   | 7500  | 0.3728          | 0.8519   | 0.7251    | 0.4323 | 0.5417 | 1.6506  |
| 0.265         | 6.0   | 9000  | 0.3836          | 0.8511   | 0.7019    | 0.4594 | 0.5554 | 1.6581  |
| 0.2284        | 7.0   | 10500 | 0.4332          | 0.8477   | 0.6611    | 0.5076 | 0.5743 | 1.6688  |
| 0.1903        | 8.0   | 12000 | 0.4834          | 0.8452   | 0.6970    | 0.4166 | 0.5215 | 1.6368  |
| 0.1527        | 9.0   | 13500 | 0.5702          | 0.8413   | 0.6809    | 0.4068 | 0.5093 | 1.6285  |
| 0.1296        | 10.0  | 15000 | 0.5942          | 0.8374   | 0.6585    | 0.4088 | 0.5044 | 1.6240  |
| 0.1158        | 11.0  | 16500 | 0.7754          | 0.8408   | 0.6680    | 0.4249 | 0.5194 | 1.6336  |
| 0.1054        | 12.0  | 18000 | 0.7936          | 0.8357   | 0.6062    | 0.5368 | 0.5694 | 1.6622  |
| 0.0879        | 13.0  | 19500 | 1.0568          | 0.8317   | 0.6971    | 0.2985 | 0.4180 | 1.5806  |
| 0.0834        | 14.0  | 21000 | 0.9730          | 0.8377   | 0.6393    | 0.4545 | 0.5313 | 1.6389  |
| 0.0744        | 15.0  | 22500 | 1.0385          | 0.8358   | 0.6390    | 0.4343 | 0.5172 | 1.6301  |
| 0.0675        | 16.0  | 24000 | 1.1625          | 0.8353   | 0.6305    | 0.4496 | 0.5249 | 1.6342  |
| 0.065         | 17.0  | 25500 | 1.2138          | 0.8325   | 0.6546    | 0.3652 | 0.4688 | 1.6034  |
| 0.0539        | 18.0  | 27000 | 1.2701          | 0.8334   | 0.6754    | 0.3409 | 0.4531 | 1.5967  |
| 0.0479        | 19.0  | 28500 | 1.2759          | 0.8367   | 0.6303    | 0.4681 | 0.5372 | 1.6420  |
| 0.0503        | 20.0  | 30000 | 1.3342          | 0.8342   | 0.6462    | 0.3993 | 0.4936 | 1.6166  |


### Framework versions

- Transformers 4.28.0
- Pytorch 2.0.1+cu118
- Datasets 2.12.0
- Tokenizers 0.13.3

{"license": "cc-by-sa-4.0", "tags": ["generated_from_trainer"], "metrics": ["accuracy", "precision", "recall", "f1"], "model-index": [{"name": "legal_bert_sm_gen1_large", "results": []}]}
transformers pytorch tensorboard bert text-classification generated_from_trainer license:cc-by-sa-4.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #tensorboard #bert #text-classification #generated_from_trainer #license-cc-by-sa-4.0 #autotrain_compatible #endpoints_compatible #region-us 

legal\_bert\_sm\_gen1\_large
============================


This model is a fine-tuned version of nlpaueb/legal-bert-base-uncased on the None dataset.
It achieves the following results on the evaluation set:


* Loss: 1.3342
* Accuracy: 0.8342
* Precision: 0.6462
* Recall: 0.3993
* F1: 0.4936
* D-index: 1.6166


Model description
-----------------


More information needed


Intended uses & limitations
---------------------------


More information needed


Training and evaluation data
----------------------------


More information needed


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 5e-05
* train\_batch\_size: 32
* eval\_batch\_size: 8
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* lr\_scheduler\_warmup\_steps: 96000
* num\_epochs: 20
* mixed\_precision\_training: Native AMP


### Training results



### Framework versions


* Transformers 4.28.0
* Pytorch 2.0.1+cu118
* Datasets 2.12.0
* Tokenizers 0.13.3



### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 5e-05
* train\_batch\_size: 32
* eval\_batch\_size: 8
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* lr\_scheduler\_warmup\_steps: 96000
* num\_epochs: 20
* mixed\_precision\_training: Native AMP ### Training results ### Framework versions


* Transformers 4.28.0
* Pytorch 2.0.1+cu118
* Datasets 2.12.0
* Tokenizers 0.13.3
TAGS
#transformers #pytorch #tensorboard #bert #text-classification #generated_from_trainer #license-cc-by-sa-4.0 #autotrain_compatible #endpoints_compatible #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 5e-05
* train\_batch\_size: 32
* eval\_batch\_size: 8
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* lr\_scheduler\_warmup\_steps: 96000
* num\_epochs: 20
* mixed\_precision\_training: Native AMP ### Training results ### Framework versions


* Transformers 4.28.0
* Pytorch 2.0.1+cu118
* Datasets 2.12.0
* Tokenizers 0.13.3