---
language:
- en
metrics:
- accuracy
library_name: transformers
pipeline_tag: text-classification
tags:
- tos
- terms of services
- bert
---

# TOSBert

**TOSBert** is a fine-tuned BERT model for sequence classification tasks. It is trained on a custom dataset for multi-label classification.

## Model Details

- **Model Name**: TOSBert
- **Model Architecture**: BERT
- **Framework**: [Hugging Face Transformers](https://huggingface.co/transformers/)
- **Model Type**: Sequence Classification (Multi-label Classification)

## Dataset

The model is trained on the [online_terms_of_service](https://huggingface.co/datasets/joelniklaus/online_terms_of_service) dataset hosted on Hugging Face. This dataset consists of text sequences extracted from various online terms of service documents. Each sequence is labeled with multiple categories related to legal and privacy terms.

## Training

The model was fine-tuned using the following parameters:

- **Number of Epochs**: 3
- **Batch Size**: 16 (both for training and evaluation)
- **Warmup Steps**: 500
- **Weight Decay**: 0.01
- **Learning Rate**: Automatically adjusted

## Usage

### Installation

To use this model, you need to install the `transformers` library from Hugging Face:

```bash
pip install transformers
```

### Loading the Model

You can load the model using the following code:

```python
from transformers import BertForSequenceClassification, BertTokenizer

model_name = "CodeHima/TOSBert"
model = BertForSequenceClassification.from_pretrained(model_name)
tokenizer = BertTokenizer.from_pretrained(model_name)
```

### Inference

Here is an example of how to use the model for inference:

```python
from transformers import pipeline

classifier = pipeline("text-classification", model=model, tokenizer=tokenizer, return_all_scores=True)

text = "Your input text here"
predictions = classifier(text)

print(predictions)
```

### Training Script

Below is an example script used for training the model:

```python
from transformers import Trainer, TrainingArguments, BertForSequenceClassification, BertTokenizer
import torch
from sklearn.metrics import accuracy_score, precision_recall_fscore_support

# Define the model
model_name = "bert-base-uncased"
model = BertForSequenceClassification.from_pretrained(model_name, num_labels=3)

# Define the tokenizer
tokenizer = BertTokenizer.from_pretrained(model_name)

# Load your dataset
# train_dataset and eval_dataset should be instances of torch.utils.data.Dataset
# Example: train_dataset = YourDataset(train_data)

# Define training arguments
training_args = TrainingArguments(
    output_dir='./results',
    num_train_epochs=3,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    warmup_steps=500,
    weight_decay=0.01,
    logging_dir='./logs',
    logging_steps=10,
    eval_strategy="epoch"
)

# Custom data collator to convert labels to floats
def data_collator(features):
    batch = {}
    first = features[0]
    if 'label' in first and first['label'] is not None:
        dtype = torch.float32
        batch['labels'] = torch.tensor([f['label'] for f in features], dtype=dtype)
    for k, v in first.items():
        if k != 'label' and v is not None and not isinstance(v, str):
            batch[k] = torch.stack([f[k] for f in features])
    return batch

# Define the compute metrics function
def compute_metrics(pred):
    labels = pred.label_ids
    preds = (pred.predictions > 0.5).astype(int)
    precision, recall, f1, _ = precision_recall_fscore_support(labels, preds, average='micro')
    acc = accuracy_score(labels, preds)
    return {
        'accuracy': acc,
        'f1': f1,
        'precision': precision,
        'recall': recall
    }

# Initialize the Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    compute_metrics=compute_metrics,
    data_collator=data_collator
)

# Train the model
trainer.train()
```

## Evaluation

To evaluate the model on the validation set, you can use the following code:

```python
results = trainer.evaluate()
print(results)
```

## License

This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for more details.

## Citation

If you use this model in your research, please cite it as follows:

```bibtex
@misc{TOSBert,
  author = {Himanshu Mohanty},
  title = {TOSBert: Fine-tuned BERT model for multi-label classification},
  year = {2024},
  publisher = {Hugging Face},
  journal = {Hugging Face Model Hub},
  howpublished = {\url{https://huggingface.co/CodeHima/TOSBert}}
}
```

## Acknowledgements

This project uses the [Hugging Face Transformers](https://huggingface.co/transformers/) library. Special thanks to the developers and contributors of this library.

```