
<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->

# bert-20news-classification

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Train Loss: 0.0479
- Train Accuracy: 0.9922
- Validation Loss: 0.2769
- Validation Accuracy: 0.9284
- Epoch: 9

## Model description

This model is a fine-tuned version of the DistilBERT model for sequence classification tasks. It was trained using Hugging Face's transformers and TensorFlow. The model expects input sequences to be tokenized according to the DistilBERT's tokenizer.

The model was trained specifically for classifying text into 20 different categories derived from the 20 Newsgroups dataset. These categories include various topics such as 'alt.atheism', 'comp.graphics', 'comp.os.ms-windows.misc', 'comp.sys.ibm.pc.hardware', 'comp.sys.mac.hardware', 'comp.windows.x', 'misc.forsale', 'rec.autos', 'rec.motorcycles', 'rec.sport.baseball', 'rec.sport.hockey', 'sci.crypt', 'sci.electronics', 'sci.med', 'sci.space', 'soc.religion.christian', 'talk.politics.guns', 'talk.politics.mideast', 'talk.politics.misc', 'talk.religion.misc'.


## Intended uses & limitations
This model is intended for classifying text into the above mentioned 20 categories. It can be used for categorizing text data from similar domains or topics.

## Training and evaluation data
the model was trained on 90% of the data from the 20 Newsgroups dataset, with the remaining 10% used for validation.

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 2120, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
- training_precision: float32

### Training results

| Train Loss | Train Accuracy | Validation Loss | Validation Accuracy | Epoch |
|:----------:|:--------------:|:---------------:|:-------------------:|:-----:|
| 1.8498     | 0.5829         | 0.9285          | 0.8012              | 0     |
| 0.6611     | 0.8406         | 0.4800          | 0.8807              | 1     |
| 0.3563     | 0.9128         | 0.3829          | 0.9002              | 2     |
| 0.2276     | 0.9475         | 0.3593          | 0.9072              | 3     |
| 0.1544     | 0.9659         | 0.3205          | 0.9214              | 4     |
| 0.1094     | 0.9779         | 0.3007          | 0.9214              | 5     |
| 0.0825     | 0.9846         | 0.2821          | 0.9258              | 6     |
| 0.0634     | 0.9895         | 0.2754          | 0.9337              | 7     |
| 0.0533     | 0.9916         | 0.2707          | 0.9337              | 8     |
| 0.0479     | 0.9922         | 0.2769          | 0.9284              | 9     |


### Framework versions

- Transformers 4.28.0
- TensorFlow 2.12.0
- Datasets 2.12.0
- Tokenizers 0.13.3

{"license": "apache-2.0", "tags": ["generated_from_keras_callback"], "model-index": [{"name": "bert-20news-classification", "results": []}]}
transformers tf distilbert text-classification generated_from_keras_callback license:apache-2.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #tf #distilbert #text-classification #generated_from_keras_callback #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 

bert-20news-classification
==========================


This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset.
It achieves the following results on the evaluation set:


* Train Loss: 0.0479
* Train Accuracy: 0.9922
* Validation Loss: 0.2769
* Validation Accuracy: 0.9284
* Epoch: 9


Model description
-----------------


This model is a fine-tuned version of the DistilBERT model for sequence classification tasks. It was trained using Hugging Face's transformers and TensorFlow. The model expects input sequences to be tokenized according to the DistilBERT's tokenizer.


The model was trained specifically for classifying text into 20 different categories derived from the 20 Newsgroups dataset. These categories include various topics such as 'alt.atheism', 'comp.graphics', 'URL', 'URL.hardware', 'URL.hardware', 'comp.windows.x', 'misc.forsale', 'URL', 'rec.motorcycles', 'URL.baseball', 'URL', 'URL', 'sci.electronics', 'URL', 'URL', 'soc.religion.christian', 'URL', 'talk.politics.mideast', 'URL', 'URL'.


Intended uses & limitations
---------------------------


This model is intended for classifying text into the above mentioned 20 categories. It can be used for categorizing text data from similar domains or topics.


Training and evaluation data
----------------------------


the model was trained on 90% of the data from the 20 Newsgroups dataset, with the remaining 10% used for validation.


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'weight\_decay': None, 'clipnorm': None, 'global\_clipnorm': None, 'clipvalue': None, 'use\_ema': False, 'ema\_momentum': 0.99, 'ema\_overwrite\_frequency': None, 'jit\_compile': True, 'is\_legacy\_optimizer': False, 'learning\_rate': {'class\_name': 'PolynomialDecay', 'config': {'initial\_learning\_rate': 2e-05, 'decay\_steps': 2120, 'end\_learning\_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
* training\_precision: float32


### Training results



### Framework versions


* Transformers 4.28.0
* TensorFlow 2.12.0
* Datasets 2.12.0
* Tokenizers 0.13.3



### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'weight\_decay': None, 'clipnorm': None, 'global\_clipnorm': None, 'clipvalue': None, 'use\_ema': False, 'ema\_momentum': 0.99, 'ema\_overwrite\_frequency': None, 'jit\_compile': True, 'is\_legacy\_optimizer': False, 'learning\_rate': {'class\_name': 'PolynomialDecay', 'config': {'initial\_learning\_rate': 2e-05, 'decay\_steps': 2120, 'end\_learning\_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.28.0
* TensorFlow 2.12.0
* Datasets 2.12.0
* Tokenizers 0.13.3
TAGS
#transformers #tf #distilbert #text-classification #generated_from_keras_callback #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'weight\_decay': None, 'clipnorm': None, 'global\_clipnorm': None, 'clipvalue': None, 'use\_ema': False, 'ema\_momentum': 0.99, 'ema\_overwrite\_frequency': None, 'jit\_compile': True, 'is\_legacy\_optimizer': False, 'learning\_rate': {'class\_name': 'PolynomialDecay', 'config': {'initial\_learning\_rate': 2e-05, 'decay\_steps': 2120, 'end\_learning\_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.28.0
* TensorFlow 2.12.0
* Datasets 2.12.0
* Tokenizers 0.13.3