library_name: transformers
license: apache-2.0
base_model: distilbert/distilbert-base-uncased
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: ai-chat-censor6
results: []
ai-chat-censor6
The primary focus of the model is detecting sexual/minors category of messages.
Possible flags are: regular, racist, underage, sexual
BEWARE
The model might categorize any talk about race as racism, for example: "Black people suffer so much in America" will be flagged as "racist".
Model also might flag any comment containing numbers below 18 as underage. This is an issue that will be addressed in next version.
Here's the next version: https://huggingface.co/andriadze/ai-chat-censor
Training and evaluation data
Model was trained on a fully synthetic dataset generated by uncensored 72b models based on qwen2.
This model is a fine-tuned version of distilbert/distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.0637
- Accuracy: 0.9903
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 6
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 0.0471 | 1.0 | 1175 | 0.0729 | 0.9854 |
| 0.0282 | 2.0 | 2350 | 0.0529 | 0.9900 |
| 0.0105 | 3.0 | 3525 | 0.0680 | 0.9888 |
| 0.0079 | 4.0 | 4700 | 0.0558 | 0.9911 |
| 0.0017 | 5.0 | 5875 | 0.0595 | 0.9902 |
| 0.0001 | 6.0 | 7050 | 0.0637 | 0.9903 |
Framework versions
- Transformers 4.44.2
- Pytorch 2.4.0+cu121
- Datasets 3.0.0
- Tokenizers 0.19.1