Edit model card

SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Model Labels

Label Examples
0
  • 'Which price points to play in?'
  • 'What are some whitespaces in terms of price bracket for Jumex in TT HM CSD?'
  • 'What are the key drivers of growth for kof in ncb?'
1
  • 'what is ROI trend for Fizzy drinks?'
  • 'Give the price vs volume comparison'
  • 'What is the change in industry mix for coca-cola in TT HM Orizaba in 2021 to 2022'

Evaluation

Metrics

Label Accuracy
all 0.8333

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("vgarg/query_type_classifier_int-e5-large_v2")
# Run inference
preds = model("How has the csd industry evolved in the last two years?")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 5 13.3684 32
Label Training Sample Count
0 40
1 36

Training Hyperparameters

  • batch_size: (16, 16)
  • num_epochs: (3, 3)
  • max_steps: -1
  • sampling_strategy: oversampling
  • num_iterations: 20
  • body_learning_rate: (2e-05, 2e-05)
  • head_learning_rate: 2e-05
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: False

Training Results

Epoch Step Training Loss Validation Loss
0.0053 1 0.3156 -
0.2632 50 0.1347 -
0.5263 100 0.0013 -
0.7895 150 0.0003 -
1.0526 200 0.0002 -
1.3158 250 0.0002 -
1.5789 300 0.0001 -
1.8421 350 0.0002 -
2.1053 400 0.0001 -
2.3684 450 0.0001 -
2.6316 500 0.0001 -
2.8947 550 0.0001 -

Framework Versions

  • Python: 3.12.2
  • SetFit: 1.0.3
  • Sentence Transformers: 2.6.1
  • Transformers: 4.39.3
  • PyTorch: 2.2.2+cpu
  • Datasets: 2.18.0
  • Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
Downloads last month
3
Safetensors
Model size
109M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for vgarg/query_type_classifier_int-e5-large_v2

Finetuned
(247)
this model

Evaluation results