
<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->

# Regression_bert_aug_CustomLoss_3

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Train Loss: 0.1282
- Train Mae: 0.3851
- Train Mse: 0.1862
- Train R2-score: 0.7249
- Validation Loss: 0.1246
- Validation Mae: 0.3798
- Validation Mse: 0.1857
- Validation R2-score: 0.8337
- Epoch: 14

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 1e-04, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
- training_precision: float32

### Training results

| Train Loss | Train Mae | Train Mse | Train R2-score | Validation Loss | Validation Mae | Validation Mse | Validation R2-score | Epoch |
|:----------:|:---------:|:---------:|:--------------:|:---------------:|:--------------:|:--------------:|:-------------------:|:-----:|
| 0.1451     | 0.4188    | 0.2732    | 0.8063         | 0.0642          | 0.3529         | 0.1994         | 0.8824              | 0     |
| 0.0567     | 0.3078    | 0.1428    | 0.7511         | 0.0452          | 0.3038         | 0.1257         | 0.8820              | 1     |
| 0.0380     | 0.2662    | 0.1007    | 0.8889         | 0.0661          | 0.2984         | 0.1442         | 0.8883              | 2     |
| 0.0363     | 0.2542    | 0.0980    | 0.8034         | 0.0318          | 0.2567         | 0.0978         | 0.9117              | 3     |
| 0.0279     | 0.2257    | 0.0714    | 0.9002         | 0.0327          | 0.2305         | 0.0793         | 0.8920              | 4     |
| 0.0241     | 0.2046    | 0.0593    | 0.8695         | 0.0306          | 0.2353         | 0.0813         | 0.9330              | 5     |
| 0.0230     | 0.1960    | 0.0540    | 0.8762         | 0.0284          | 0.2160         | 0.0710         | 0.9197              | 6     |
| 0.0223     | 0.1914    | 0.0510    | 0.9366         | 0.0285          | 0.2251         | 0.0791         | 0.9282              | 7     |
| 0.0223     | 0.1923    | 0.0516    | 0.9498         | 0.0306          | 0.2042         | 0.0748         | 0.9309              | 8     |
| 0.0231     | 0.1827    | 0.0493    | 0.8516         | 0.0302          | 0.2009         | 0.0682         | 0.9198              | 9     |
| 0.0335     | 0.1794    | 0.0576    | 0.9259         | 0.0765          | 0.3684         | 0.2192         | 0.8243              | 10    |
| 0.1380     | 0.3960    | 0.2567    | 0.8748         | 0.1037          | 0.4172         | 0.2244         | 0.6992              | 11    |
| 0.1078     | 0.4071    | 0.2170    | 0.8256         | 0.1219          | 0.4020         | 0.2234         | 0.7304              | 12    |
| 0.1217     | 0.3807    | 0.2060    | 0.8084         | 0.1434          | 0.3934         | 0.2113         | 0.8317              | 13    |
| 0.1282     | 0.3851    | 0.1862    | 0.7249         | 0.1246          | 0.3798         | 0.1857         | 0.8337              | 14    |


### Framework versions

- Transformers 4.28.1
- TensorFlow 2.12.0
- Datasets 2.12.0
- Tokenizers 0.13.3

{"license": "apache-2.0", "tags": ["generated_from_keras_callback"], "model-index": [{"name": "Regression_bert_aug_CustomLoss_3", "results": []}]}
transformers tf distilbert text-classification generated_from_keras_callback license:apache-2.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #tf #distilbert #text-classification #generated_from_keras_callback #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 

Regression\_bert\_aug\_CustomLoss\_3
====================================


This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset.
It achieves the following results on the evaluation set:


* Train Loss: 0.1282
* Train Mae: 0.3851
* Train Mse: 0.1862
* Train R2-score: 0.7249
* Validation Loss: 0.1246
* Validation Mae: 0.3798
* Validation Mse: 0.1857
* Validation R2-score: 0.8337
* Epoch: 14


Model description
-----------------


More information needed


Intended uses & limitations
---------------------------


More information needed


Training and evaluation data
----------------------------


More information needed


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'weight\_decay': None, 'clipnorm': None, 'global\_clipnorm': None, 'clipvalue': None, 'use\_ema': False, 'ema\_momentum': 0.99, 'ema\_overwrite\_frequency': None, 'jit\_compile': True, 'is\_legacy\_optimizer': False, 'learning\_rate': 1e-04, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
* training\_precision: float32


### Training results



### Framework versions


* Transformers 4.28.1
* TensorFlow 2.12.0
* Datasets 2.12.0
* Tokenizers 0.13.3



### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'weight\_decay': None, 'clipnorm': None, 'global\_clipnorm': None, 'clipvalue': None, 'use\_ema': False, 'ema\_momentum': 0.99, 'ema\_overwrite\_frequency': None, 'jit\_compile': True, 'is\_legacy\_optimizer': False, 'learning\_rate': 1e-04, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.28.1
* TensorFlow 2.12.0
* Datasets 2.12.0
* Tokenizers 0.13.3
TAGS
#transformers #tf #distilbert #text-classification #generated_from_keras_callback #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'weight\_decay': None, 'clipnorm': None, 'global\_clipnorm': None, 'clipvalue': None, 'use\_ema': False, 'ema\_momentum': 0.99, 'ema\_overwrite\_frequency': None, 'jit\_compile': True, 'is\_legacy\_optimizer': False, 'learning\_rate': 1e-04, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.28.1
* TensorFlow 2.12.0
* Datasets 2.12.0
* Tokenizers 0.13.3