
This repo has an optimized version of [Detoxify](https://github.com/unitaryai/detoxify/), which needs less disk space and less memory at the cost of just a little bit of accuracy.

This is an experiment for me to learn how to use [🤗 Optimum](https://huggingface.co/docs/optimum/index).

# Usage

Loading the model requires the [🤗 Optimum](https://huggingface.co/docs/optimum/index) library installed.

```python
from optimum.onnxruntime import ORTModelForSequenceClassification
from optimum.pipelines import pipeline as opt_pipeline
from transformers import AutoTokenizer


tokenizer = AutoTokenizer.from_pretrained("dcferreira/detoxify-optimized")
model = ORTModelForSequenceClassification.from_pretrained("dcferreira/detoxify-optimized")
pipe = opt_pipeline(
    model=model,
    task="text-classification",
    function_to_apply="sigmoid",
    accelerator="ort",
    tokenizer=tokenizer,
    top_k=None,  # return scores for all the labels, model was trained as multilabel
)

print(pipe(['example text','exemple de texte','texto de ejemplo','testo di esempio','texto de exemplo','örnek metin','пример текста']))
```

# Performance

The table below compares some statistics on running the original model, vs the original model with the [onnxruntime](https://onnxruntime.ai/), vs optimizing the model with onnxruntime.


| model          | Accuracy (%) | Samples p/ second (CPU) | Samples p/ second (GPU) | GPU VRAM | Disk Space |
|----------------|----------|-------------------------|-------------------------|----------|------------|
| original       | 92.1083  | 16                      | 250                     | 3GB      | 1.1GB      |
| ort            | 92.1067  | 19                      | 340                     | 4GB      | 1.1GB      |
| optimized (O4) | 92.1031  | 14                      | 650                     | 2GB      | 540MB      |

For details on how these numbers were reached, check out `evaluate.py` in this repo.

{"license": "apache-2.0"}
transformers onnx xlm-roberta text-classification license:apache-2.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #onnx #xlm-roberta #text-classification #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 

This repo has an optimized version of Detoxify, which needs less disk space and less memory at the cost of just a little bit of accuracy.


This is an experiment for me to learn how to use Optimum.


Usage
=====


Loading the model requires the Optimum library installed.


Performance
===========


The table below compares some statistics on running the original model, vs the original model with the onnxruntime, vs optimizing the model with onnxruntime.



For details on how these numbers were reached, check out 'URL' in this repo.



TAGS
#transformers #onnx #xlm-roberta #text-classification #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 
