Text Equivalence Classification

This model is a fine-tuned version of distilroberta-base specifically for the task of determining whether two sentences are equivalent in meaning. It was trained on a curated dataset combining multiple sources to cover a wide range of topics and styles.

Model Description

The model aims to capture the nuanced relationships between sentences, leveraging the powerful pre-trained DistilRoBERTa base for understanding contextual meanings. It has been fine-tuned with a specific focus on identifying semantic equivalence, making it suitable for applications such as paraphrase identification, duplicate question detection, and semantic search tasks.

Intended Uses & Limitations

This model is intended for use in applications requiring the determination of semantic equivalence between pairs of sentences. It can be utilized in creating more efficient search algorithms, enhancing chatbot understandings, and improving content moderation by identifying duplicate or similar content.

However, the model may exhibit limitations when dealing with sentences that contain highly domain-specific jargon, rare languages, or nuanced cultural references. It is recommended to test the model with your specific data and use case to ensure its effectiveness.

Training and Evaluation Data

The model was trained on a dataset named "DatasetX," which is a composite of several public datasets including portions of the Quora Question Pairs and the Microsoft Research Paraphrase Corpus. This combination was chosen to create a robust model capable of understanding a wide variety of sentence structures and meanings.

Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

Learning rate: 5e-05
Train batch size: 8
Eval batch size: 8
Seed: 42
Optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
LR scheduler type: linear
Number of epochs: 3

Training Results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.5396	1.09	500	0.5511	0.8309	0.8848
0.3722	2.18	1000	0.4973	0.8529	0.8944

Framework Versions

Transformers 4.37.2
Pytorch 2.2.0
Datasets 2.17.0
Tokenizers 0.15.2

IgnaCodeIA
/

text-Classification