---
library_name: transformers
license: apache-2.0
base_model: distilbert/distilbert-base-uncased
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: ai-chat-censor6
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# ai-chat-censor6

The primary focus of the model is detecting sexual/minors category of messages.

Possible flags are: regular, racist, underage, sexual


# BEWARE

The model might categorize any talk about race as racism, for example: "Black people suffer so much in America" will be flagged as "racist".

Model also might flag any comment containing numbers below 18 as underage. This is an issue that will be addressed in next version.

Here's the next version:
https://huggingface.co/andriadze/ai-chat-censor


## Training and evaluation data

Model was trained on a fully synthetic dataset generated by uncensored 72b models based on qwen2.

This model is a fine-tuned version of [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0637
- Accuracy: 0.9903


### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 6

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 0.0471        | 1.0   | 1175 | 0.0729          | 0.9854   |
| 0.0282        | 2.0   | 2350 | 0.0529          | 0.9900   |
| 0.0105        | 3.0   | 3525 | 0.0680          | 0.9888   |
| 0.0079        | 4.0   | 4700 | 0.0558          | 0.9911   |
| 0.0017        | 5.0   | 5875 | 0.0595          | 0.9902   |
| 0.0001        | 6.0   | 7050 | 0.0637          | 0.9903   |


### Framework versions

- Transformers 4.44.2
- Pytorch 2.4.0+cu121
- Datasets 3.0.0
- Tokenizers 0.19.1