---
library_name: setfit
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
metrics:
- metric
widget:
- text: A combined 20 million people per year die of smoking and hunger, so authorities
can't seem to feed people and they allow you to buy cigarettes but we are facing
another lockdown for a virus that has a 99.5% survival rate!!! THINK PEOPLE. LOOK
AT IT LOGICALLY WITH YOUR OWN EYES.
- text: Scientists do not agree on the consequences of climate change, nor is there
any consensus on that subject. The predictions on that from are just ascientific
speculation. Bring on the warming."
- text: If Tam is our "top doctor"....I am going back to leaches and voodoo...just
as much science in that as the crap she spouts
- text: "Can she skip school by herself and sit infront of parliament? \r\n Fake emotions\
\ and just a good actor."
- text: my dad had huge ones..so they may be real..
pipeline_tag: text-classification
inference: true
base_model: sentence-transformers/paraphrase-mpnet-base-v2
model-index:
- name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: Unknown
type: unknown
split: test
metrics:
- type: metric
value: 0.4482758620689655
name: Metric
---
# SetFit with sentence-transformers/paraphrase-mpnet-base-v2
This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.
## Model Details
### Model Description
- **Model Type:** SetFit
- **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
- **Maximum Sequence Length:** 512 tokens
- **Number of Classes:** 9 classes
### Model Sources
- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
### Model Labels
| Label | Examples |
|:------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 5 | - 'please show us the evidence I asked for'
- 'Please follow https://youtu.be/WpTCt-S-qLM ??'
- 'Answer the question. The first is illegal in NY, the second is legal.'
|
| 0 | - 'This guy sounds like he needs to clear his throat'
- 'If only most senates in the US can see how climate change can not only effect our planets environment, it also our economys like theirs.'
- '1st Comment!'
|
| 7 | - 'Yeah I can do that.'
- "I can totally accept that the government fund green energy, it's for the best."
- 'Yes, he is right ! My Dr. did exactly what he is saying. He started antibiotics then 5 days started the steroids. Hopefully other dr will do the same.'
|
| 2 | - 'ok boomer'
- 'I can see where ur coming from she did invite this cousin to live there which mind you is likely an act of kindness, and it is ur private space. But tbh you are also being extremely petty, this man is on the couch, he has early morning shifts, using ur bathroom would not disturb a single person, while using ur roommates means this man has to be very uncomfortable walk through a sleeping persons room every morning and sneak back out again. Likely waking up the cousin in the process. Thats why I believe its an Everyone Sucks here cause theyre both asshole like moves.'
- 'suggesting the vaccine to women who are pregnant, when it can cause for many women earlier heavier longer periods, means it triggers a period, we know women in our personal lives that have experienced that along with their period coming twice that month, and we know to avoid foods that can trigger a period because that could potentially cause a miscarriage, so why suggest it?? this is how the whole world will lose confidence in science and medicine because they can down right lie to our face claiming they understand more then you meanwhile propagating the agenda of pharmaceutical companies. absaloutly disgusting and shame on you for betraying our trust.'
|
| 3 | - 'Nothing is hotter than Shawn, not even the sun mate??'
- 'No Im not trolling.'
- "I'm not being hurtful. I'm being honest. You need to vaccinate your cat"
|
| 6 | - 'So youre taking a government course?'
- 'Is this journalist on work experience ?'
- 'Who is here after the movie'
|
| 4 | - 'Forward looking required now, which the leaders are doing and doing their best, sleep deprived and a world of responsibility on them. Thanks to all'
- "Oh, great, you could do that? That'd help me out really."
- "Couldn't be more proud and happy that these heroes are finally taking a stand to the horrible ways the current government is pushing the country"
|
| 1 | - "Maybe STOP paying commanders who have NEVER even picked up a hose and give the power back to the brigades CAPTAIN'S. And things will start to get better, from someone who is actually doing something. Snowy mountains Australia."
- "@Keith Bawden You have an education in science? Me too! Did you study in any field relevant to the topic? I am currently doing a PhD in biogeochemistry, and my research group is involved in climate science. I can tell you the VAST majority of scientists in fields directly related to or peripheral to climate change accept that it is indeed a real phenomenon, and it is caused by humans. The exceptions you can name are exactly that, exceptions. I'll grant you, Zarkhova may be right about a coming grand solar minimum, but even if so, all it would do is slightly slow temperature increase. There would be no mini ice age (Fuelner and Rahmstorf, 2010). The question is, in 2-3 years, when a 'mini ice age' does not occur, will you change your mind, or find some other reason to deny?"
- "You should have gotten herd immunity in Changi, considering 95% efficacy of Pfizer and 80% or more are vaccinated.\r\nIt's either the efficacy is faked or the vaccine is useless against the indian variant."
|
| 8 | - "Things got out of hand, I'm sorry."
- 'Oh no, I did not mean it that way, it was completely misunderstood what I was saying. Didnt mean to offend you, sorry!'
- 'Sorry.'
|
## Evaluation
### Metrics
| Label | Metric |
|:--------|:-------|
| **all** | 0.4483 |
## Uses
### Direct Use for Inference
First install the SetFit library:
```bash
pip install setfit
```
Then you can load this model and run inference.
```python
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("CrisisNarratives/setfit-9classes-single_label")
# Run inference
preds = model("my dad had huge ones..so they may be real..")
```
## Training Details
### Training Set Metrics
| Training set | Min | Median | Max |
|:-------------|:----|:--------|:-----|
| Word count | 1 | 25.8891 | 1681 |
| Label | Training Sample Count |
|:------|:----------------------|
| 0 | 156 |
| 1 | 81 |
| 2 | 64 |
| 3 | 52 |
| 4 | 46 |
| 5 | 63 |
| 6 | 35 |
| 7 | 37 |
| 8 | 7 |
### Training Hyperparameters
- batch_size: (16, 16)
- num_epochs: (3, 3)
- max_steps: -1
- sampling_strategy: oversampling
- num_iterations: 40
- body_learning_rate: (1.752e-05, 1.752e-05)
- head_learning_rate: 1.752e-05
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 30
- eval_max_steps: -1
- load_best_model_at_end: False
### Training Results
| Epoch | Step | Training Loss | Validation Loss |
|:------:|:-----:|:-------------:|:---------------:|
| 0.0004 | 1 | 0.3913 | - |
| 0.0185 | 50 | 0.3901 | - |
| 0.0370 | 100 | 0.219 | - |
| 0.0555 | 150 | 0.2308 | - |
| 0.0739 | 200 | 0.2161 | - |
| 0.0924 | 250 | 0.2 | - |
| 0.1109 | 300 | 0.2436 | - |
| 0.1294 | 350 | 0.2219 | - |
| 0.1479 | 400 | 0.1266 | - |
| 0.1664 | 450 | 0.1043 | - |
| 0.1848 | 500 | 0.076 | - |
| 0.2033 | 550 | 0.1331 | - |
| 0.2218 | 600 | 0.0858 | - |
| 0.2403 | 650 | 0.0355 | - |
| 0.2588 | 700 | 0.0475 | - |
| 0.2773 | 750 | 0.066 | - |
| 0.2957 | 800 | 0.0667 | - |
| 0.3142 | 850 | 0.0082 | - |
| 0.3327 | 900 | 0.0658 | - |
| 0.3512 | 950 | 0.0042 | - |
| 0.3697 | 1000 | 0.095 | - |
| 0.3882 | 1050 | 0.0598 | - |
| 0.4067 | 1100 | 0.0037 | - |
| 0.4251 | 1150 | 0.0155 | - |
| 0.4436 | 1200 | 0.0028 | - |
| 0.4621 | 1250 | 0.0025 | - |
| 0.4806 | 1300 | 0.0542 | - |
| 0.4991 | 1350 | 0.001 | - |
| 0.5176 | 1400 | 0.0056 | - |
| 0.5360 | 1450 | 0.001 | - |
| 0.5545 | 1500 | 0.0011 | - |
| 0.5730 | 1550 | 0.0007 | - |
| 0.5915 | 1600 | 0.0014 | - |
| 0.6100 | 1650 | 0.0018 | - |
| 0.6285 | 1700 | 0.0012 | - |
| 0.6470 | 1750 | 0.0005 | - |
| 0.6654 | 1800 | 0.0006 | - |
| 0.6839 | 1850 | 0.0003 | - |
| 0.7024 | 1900 | 0.0002 | - |
| 0.7209 | 1950 | 0.0044 | - |
| 0.7394 | 2000 | 0.003 | - |
| 0.7579 | 2050 | 0.0005 | - |
| 0.7763 | 2100 | 0.0006 | - |
| 0.7948 | 2150 | 0.0005 | - |
| 0.8133 | 2200 | 0.0002 | - |
| 0.8318 | 2250 | 0.0003 | - |
| 0.8503 | 2300 | 0.0003 | - |
| 0.8688 | 2350 | 0.0006 | - |
| 0.8872 | 2400 | 0.0002 | - |
| 0.9057 | 2450 | 0.002 | - |
| 0.9242 | 2500 | 0.0003 | - |
| 0.9427 | 2550 | 0.0002 | - |
| 0.9612 | 2600 | 0.0009 | - |
| 0.9797 | 2650 | 0.0001 | - |
| 0.9982 | 2700 | 0.0002 | - |
| 1.0166 | 2750 | 0.0003 | - |
| 1.0351 | 2800 | 0.0003 | - |
| 1.0536 | 2850 | 0.0004 | - |
| 1.0721 | 2900 | 0.0003 | - |
| 1.0906 | 2950 | 0.0004 | - |
| 1.1091 | 3000 | 0.0003 | - |
| 1.1275 | 3050 | 0.0001 | - |
| 1.1460 | 3100 | 0.0002 | - |
| 1.1645 | 3150 | 0.0005 | - |
| 1.1830 | 3200 | 0.0004 | - |
| 1.2015 | 3250 | 0.0003 | - |
| 1.2200 | 3300 | 0.0003 | - |
| 1.2384 | 3350 | 0.0004 | - |
| 1.2569 | 3400 | 0.0003 | - |
| 1.2754 | 3450 | 0.0002 | - |
| 1.2939 | 3500 | 0.0002 | - |
| 1.3124 | 3550 | 0.0003 | - |
| 1.3309 | 3600 | 0.0005 | - |
| 1.3494 | 3650 | 0.0002 | - |
| 1.3678 | 3700 | 0.0003 | - |
| 1.3863 | 3750 | 0.0002 | - |
| 1.4048 | 3800 | 0.0001 | - |
| 1.4233 | 3850 | 0.0001 | - |
| 1.4418 | 3900 | 0.0004 | - |
| 1.4603 | 3950 | 0.0001 | - |
| 1.4787 | 4000 | 0.0002 | - |
| 1.4972 | 4050 | 0.001 | - |
| 1.5157 | 4100 | 0.0002 | - |
| 1.5342 | 4150 | 0.0003 | - |
| 1.5527 | 4200 | 0.0001 | - |
| 1.5712 | 4250 | 0.0001 | - |
| 1.5896 | 4300 | 0.0002 | - |
| 1.6081 | 4350 | 0.0005 | - |
| 1.6266 | 4400 | 0.0001 | - |
| 1.6451 | 4450 | 0.0002 | - |
| 1.6636 | 4500 | 0.0001 | - |
| 1.6821 | 4550 | 0.0001 | - |
| 1.7006 | 4600 | 0.0001 | - |
| 1.7190 | 4650 | 0.0001 | - |
| 1.7375 | 4700 | 0.0001 | - |
| 1.7560 | 4750 | 0.0002 | - |
| 1.7745 | 4800 | 0.0001 | - |
| 1.7930 | 4850 | 0.0001 | - |
| 1.8115 | 4900 | 0.0001 | - |
| 1.8299 | 4950 | 0.0 | - |
| 1.8484 | 5000 | 0.0001 | - |
| 1.8669 | 5050 | 0.0001 | - |
| 1.8854 | 5100 | 0.0001 | - |
| 1.9039 | 5150 | 0.0001 | - |
| 1.9224 | 5200 | 0.0001 | - |
| 1.9409 | 5250 | 0.0001 | - |
| 1.9593 | 5300 | 0.0001 | - |
| 1.9778 | 5350 | 0.0 | - |
| 1.9963 | 5400 | 0.0001 | - |
| 2.0148 | 5450 | 0.0001 | - |
| 2.0333 | 5500 | 0.0001 | - |
| 2.0518 | 5550 | 0.0001 | - |
| 2.0702 | 5600 | 0.0002 | - |
| 2.0887 | 5650 | 0.0001 | - |
| 2.1072 | 5700 | 0.0001 | - |
| 2.1257 | 5750 | 0.0 | - |
| 2.1442 | 5800 | 0.0001 | - |
| 2.1627 | 5850 | 0.0001 | - |
| 2.1811 | 5900 | 0.0003 | - |
| 2.1996 | 5950 | 0.0001 | - |
| 2.2181 | 6000 | 0.0002 | - |
| 2.2366 | 6050 | 0.0001 | - |
| 2.2551 | 6100 | 0.0001 | - |
| 2.2736 | 6150 | 0.0001 | - |
| 2.2921 | 6200 | 0.0001 | - |
| 2.3105 | 6250 | 0.0001 | - |
| 2.3290 | 6300 | 0.0001 | - |
| 2.3475 | 6350 | 0.0001 | - |
| 2.3660 | 6400 | 0.0001 | - |
| 2.3845 | 6450 | 0.0001 | - |
| 2.4030 | 6500 | 0.0001 | - |
| 2.4214 | 6550 | 0.0001 | - |
| 2.4399 | 6600 | 0.0001 | - |
| 2.4584 | 6650 | 0.0001 | - |
| 2.4769 | 6700 | 0.0001 | - |
| 2.4954 | 6750 | 0.0001 | - |
| 2.5139 | 6800 | 0.0001 | - |
| 2.5323 | 6850 | 0.0002 | - |
| 2.5508 | 6900 | 0.0001 | - |
| 2.5693 | 6950 | 0.0002 | - |
| 2.5878 | 7000 | 0.0001 | - |
| 2.6063 | 7050 | 0.0001 | - |
| 2.6248 | 7100 | 0.0 | - |
| 2.6433 | 7150 | 0.0 | - |
| 2.6617 | 7200 | 0.0001 | - |
| 2.6802 | 7250 | 0.0001 | - |
| 2.6987 | 7300 | 0.0002 | - |
| 2.7172 | 7350 | 0.0001 | - |
| 2.7357 | 7400 | 0.0001 | - |
| 2.7542 | 7450 | 0.0002 | - |
| 2.7726 | 7500 | 0.0 | - |
| 2.7911 | 7550 | 0.0001 | - |
| 2.8096 | 7600 | 0.0005 | - |
| 2.8281 | 7650 | 0.0001 | - |
| 2.8466 | 7700 | 0.0001 | - |
| 2.8651 | 7750 | 0.0001 | - |
| 2.8835 | 7800 | 0.0002 | - |
| 2.9020 | 7850 | 0.0 | - |
| 2.9205 | 7900 | 0.0001 | - |
| 2.9390 | 7950 | 0.0 | - |
| 2.9575 | 8000 | 0.0001 | - |
| 2.9760 | 8050 | 0.0001 | - |
| 2.9945 | 8100 | 0.0001 | - |
| 0.0002 | 1 | 0.0001 | - |
| 0.0108 | 50 | 0.0003 | - |
| 0.0216 | 100 | 0.0001 | - |
| 0.0323 | 150 | 0.0004 | - |
| 0.0431 | 200 | 0.0002 | - |
| 0.0539 | 250 | 0.0006 | - |
| 0.0647 | 300 | 0.0001 | - |
| 0.0755 | 350 | 0.0002 | - |
| 0.0862 | 400 | 0.0051 | - |
| 0.0970 | 450 | 0.1866 | - |
| 0.1078 | 500 | 0.11 | - |
| 0.1186 | 550 | 0.1214 | - |
| 0.1294 | 600 | 0.2073 | - |
| 0.1401 | 650 | 0.019 | - |
| 0.1509 | 700 | 0.0762 | - |
| 0.1617 | 750 | 0.1901 | - |
| 0.1725 | 800 | 0.1234 | - |
| 0.1833 | 850 | 0.0601 | - |
| 0.1940 | 900 | 0.4192 | - |
| 0.2048 | 950 | 0.0397 | - |
| 0.2156 | 1000 | 0.111 | - |
| 0.2264 | 1050 | 0.055 | - |
| 0.2372 | 1100 | 0.0146 | - |
| 0.2480 | 1150 | 0.1277 | - |
| 0.2587 | 1200 | 0.0236 | - |
| 0.2695 | 1250 | 0.0087 | - |
| 0.2803 | 1300 | 0.2315 | - |
| 0.2911 | 1350 | 0.3547 | - |
| 0.3019 | 1400 | 0.5957 | - |
| 0.3126 | 1450 | 0.2253 | - |
| 0.3234 | 1500 | 0.2068 | - |
| 0.3342 | 1550 | 0.3203 | - |
| 0.3450 | 1600 | 0.5608 | - |
| 0.3558 | 1650 | 0.3014 | - |
| 0.3665 | 1700 | 0.3287 | - |
| 0.3773 | 1750 | 0.3206 | - |
| 0.3881 | 1800 | 0.4245 | - |
| 0.3989 | 1850 | 0.2641 | - |
| 0.4097 | 1900 | 0.4057 | - |
| 0.4204 | 1950 | 0.3891 | - |
| 0.4312 | 2000 | 0.3688 | - |
| 0.4420 | 2050 | 0.4642 | - |
| 0.4528 | 2100 | 0.3684 | - |
| 0.4636 | 2150 | 0.246 | - |
| 0.4743 | 2200 | 0.177 | - |
| 0.4851 | 2250 | 0.3416 | - |
| 0.4959 | 2300 | 0.3931 | - |
| 0.5067 | 2350 | 0.2617 | - |
| 0.5175 | 2400 | 0.5679 | - |
| 0.5282 | 2450 | 0.3879 | - |
| 0.5390 | 2500 | 0.3916 | - |
| 0.5498 | 2550 | 0.3657 | - |
| 0.5606 | 2600 | 0.3382 | - |
| 0.5714 | 2650 | 0.4621 | - |
| 0.5821 | 2700 | 0.3235 | - |
| 0.5929 | 2750 | 0.2986 | - |
| 0.6037 | 2800 | 0.3001 | - |
| 0.6145 | 2850 | 0.2309 | - |
| 0.6253 | 2900 | 0.1802 | - |
| 0.6361 | 2950 | 0.2648 | - |
| 0.6468 | 3000 | 0.2875 | - |
| 0.6576 | 3050 | 0.2888 | - |
| 0.6684 | 3100 | 0.2563 | - |
| 0.6792 | 3150 | 0.3129 | - |
| 0.6900 | 3200 | 0.2229 | - |
| 0.7007 | 3250 | 0.2462 | - |
| 0.7115 | 3300 | 0.283 | - |
| 0.7223 | 3350 | 0.3622 | - |
| 0.7331 | 3400 | 0.3428 | - |
| 0.7439 | 3450 | 0.4274 | - |
| 0.7546 | 3500 | 0.4131 | - |
| 0.7654 | 3550 | 0.2123 | - |
| 0.7762 | 3600 | 0.326 | - |
| 0.7870 | 3650 | 0.2488 | - |
| 0.7978 | 3700 | 0.4046 | - |
| 0.8085 | 3750 | 0.2664 | - |
| 0.8193 | 3800 | 0.2426 | - |
| 0.8301 | 3850 | 0.3534 | - |
| 0.8409 | 3900 | 0.2753 | - |
| 0.8517 | 3950 | 0.3177 | - |
| 0.8624 | 4000 | 0.222 | - |
| 0.8732 | 4050 | 0.3942 | - |
| 0.8840 | 4100 | 0.1932 | - |
| 0.8948 | 4150 | 0.2727 | - |
| 0.9056 | 4200 | 0.2713 | - |
| 0.9163 | 4250 | 0.3888 | - |
| 0.9271 | 4300 | 0.3155 | - |
| 0.9379 | 4350 | 0.2727 | - |
| 0.9487 | 4400 | 0.4148 | - |
| 0.9595 | 4450 | 0.297 | - |
| 0.9702 | 4500 | 0.2154 | - |
| 0.9810 | 4550 | 0.2617 | - |
| 0.9918 | 4600 | 0.255 | - |
| 1.0026 | 4650 | 0.395 | - |
| 1.0134 | 4700 | 0.4104 | - |
| 1.0241 | 4750 | 0.2675 | - |
| 1.0349 | 4800 | 0.2458 | - |
| 1.0457 | 4850 | 0.316 | - |
| 1.0565 | 4900 | 0.3786 | - |
| 1.0673 | 4950 | 0.2206 | - |
| 1.0781 | 5000 | 0.3946 | - |
| 1.0888 | 5050 | 0.2178 | - |
| 1.0996 | 5100 | 0.302 | - |
| 1.1104 | 5150 | 0.2449 | - |
| 1.1212 | 5200 | 0.2644 | - |
| 1.1320 | 5250 | 0.3111 | - |
| 1.1427 | 5300 | 0.3953 | - |
| 1.1535 | 5350 | 0.2064 | - |
| 1.1643 | 5400 | 0.3149 | - |
| 1.1751 | 5450 | 0.2073 | - |
| 1.1859 | 5500 | 0.3759 | - |
| 1.1966 | 5550 | 0.2044 | - |
| 1.2074 | 5600 | 0.2034 | - |
| 1.2182 | 5650 | 0.2325 | - |
| 1.2290 | 5700 | 0.2393 | - |
| 1.2398 | 5750 | 0.3568 | - |
| 1.2505 | 5800 | 0.2234 | - |
| 1.2613 | 5850 | 0.2428 | - |
| 1.2721 | 5900 | 0.3561 | - |
| 1.2829 | 5950 | 0.1885 | - |
| 1.2937 | 6000 | 0.3153 | - |
| 1.3044 | 6050 | 0.2288 | - |
| 1.3152 | 6100 | 0.2852 | - |
| 1.3260 | 6150 | 0.289 | - |
| 1.3368 | 6200 | 0.3719 | - |
| 1.3476 | 6250 | 0.1921 | - |
| 1.3583 | 6300 | 0.266 | - |
| 1.3691 | 6350 | 0.2743 | - |
| 1.3799 | 6400 | 0.3637 | - |
| 1.3907 | 6450 | 0.3849 | - |
| 1.4015 | 6500 | 0.1926 | - |
| 1.4122 | 6550 | 0.3594 | - |
| 1.4230 | 6600 | 0.3263 | - |
| 1.4338 | 6650 | 0.4645 | - |
| 1.4446 | 6700 | 0.2304 | - |
| 1.4554 | 6750 | 0.2337 | - |
| 1.4661 | 6800 | 0.2812 | - |
| 1.4769 | 6850 | 0.2975 | - |
| 1.4877 | 6900 | 0.4025 | - |
| 1.4985 | 6950 | 0.1897 | - |
| 1.5093 | 7000 | 0.4523 | - |
| 1.5201 | 7050 | 0.1906 | - |
| 1.5308 | 7100 | 0.2756 | - |
| 1.5416 | 7150 | 0.3313 | - |
| 1.5524 | 7200 | 0.2999 | - |
| 1.5632 | 7250 | 0.2517 | - |
| 1.5740 | 7300 | 0.2421 | - |
| 1.5847 | 7350 | 0.2864 | - |
| 1.5955 | 7400 | 0.3119 | - |
| 1.6063 | 7450 | 0.2178 | - |
| 1.6171 | 7500 | 0.4006 | - |
| 1.6279 | 7550 | 0.2744 | - |
| 1.6386 | 7600 | 0.2306 | - |
| 1.6494 | 7650 | 0.2772 | - |
| 1.6602 | 7700 | 0.2103 | - |
| 1.6710 | 7750 | 0.3151 | - |
| 1.6818 | 7800 | 0.3457 | - |
| 1.6925 | 7850 | 0.2146 | - |
| 1.7033 | 7900 | 0.2105 | - |
| 1.7141 | 7950 | 0.1986 | - |
| 1.7249 | 8000 | 0.2604 | - |
| 1.7357 | 8050 | 0.1683 | - |
| 1.7464 | 8100 | 0.2814 | - |
| 1.7572 | 8150 | 0.2088 | - |
| 1.7680 | 8200 | 0.3935 | - |
| 1.7788 | 8250 | 0.3019 | - |
| 1.7896 | 8300 | 0.3094 | - |
| 1.8003 | 8350 | 0.2024 | - |
| 1.8111 | 8400 | 0.2901 | - |
| 1.8219 | 8450 | 0.2392 | - |
| 1.8327 | 8500 | 0.3296 | - |
| 1.8435 | 8550 | 0.2818 | - |
| 1.8542 | 8600 | 0.2898 | - |
| 1.8650 | 8650 | 0.2598 | - |
| 1.8758 | 8700 | 0.3531 | - |
| 1.8866 | 8750 | 0.2989 | - |
| 1.8974 | 8800 | 0.2356 | - |
| 1.9082 | 8850 | 0.3657 | - |
| 1.9189 | 8900 | 0.3765 | - |
| 1.9297 | 8950 | 0.2668 | - |
| 1.9405 | 9000 | 0.4219 | - |
| 1.9513 | 9050 | 0.3362 | - |
| 1.9621 | 9100 | 0.325 | - |
| 1.9728 | 9150 | 0.267 | - |
| 1.9836 | 9200 | 0.2945 | - |
| 1.9944 | 9250 | 0.2129 | - |
| 2.0052 | 9300 | 0.351 | - |
| 2.0160 | 9350 | 0.4508 | - |
| 2.0267 | 9400 | 0.2375 | - |
| 2.0375 | 9450 | 0.2669 | - |
| 2.0483 | 9500 | 0.232 | - |
| 2.0591 | 9550 | 0.2469 | - |
| 2.0699 | 9600 | 0.2644 | - |
| 2.0806 | 9650 | 0.239 | - |
| 2.0914 | 9700 | 0.3189 | - |
| 2.1022 | 9750 | 0.2711 | - |
| 2.1130 | 9800 | 0.2627 | - |
| 2.1238 | 9850 | 0.2213 | - |
| 2.1345 | 9900 | 0.2311 | - |
| 2.1453 | 9950 | 0.3009 | - |
| 2.1561 | 10000 | 0.2068 | - |
| 2.1669 | 10050 | 0.3129 | - |
| 2.1777 | 10100 | 0.2901 | - |
| 2.1884 | 10150 | 0.2743 | - |
| 2.1992 | 10200 | 0.2809 | - |
| 2.2100 | 10250 | 0.249 | - |
| 2.2208 | 10300 | 0.3017 | - |
| 2.2316 | 10350 | 0.4271 | - |
| 2.2423 | 10400 | 0.2551 | - |
| 2.2531 | 10450 | 0.1911 | - |
| 2.2639 | 10500 | 0.3297 | - |
| 2.2747 | 10550 | 0.3251 | - |
| 2.2855 | 10600 | 0.267 | - |
| 2.2962 | 10650 | 0.3022 | - |
| 2.3070 | 10700 | 0.2353 | - |
| 2.3178 | 10750 | 0.3533 | - |
| 2.3286 | 10800 | 0.216 | - |
| 2.3394 | 10850 | 0.3003 | - |
| 2.3502 | 10900 | 0.2943 | - |
| 2.3609 | 10950 | 0.2959 | - |
| 2.3717 | 11000 | 0.3203 | - |
| 2.3825 | 11050 | 0.2962 | - |
| 2.3933 | 11100 | 0.2475 | - |
| 2.4041 | 11150 | 0.2933 | - |
| 2.4148 | 11200 | 0.2903 | - |
| 2.4256 | 11250 | 0.328 | - |
| 2.4364 | 11300 | 0.1893 | - |
| 2.4472 | 11350 | 0.2367 | - |
| 2.4580 | 11400 | 0.2473 | - |
| 2.4687 | 11450 | 0.2751 | - |
| 2.4795 | 11500 | 0.2708 | - |
| 2.4903 | 11550 | 0.3104 | - |
| 2.5011 | 11600 | 0.2791 | - |
| 2.5119 | 11650 | 0.3181 | - |
| 2.5226 | 11700 | 0.2411 | - |
| 2.5334 | 11750 | 0.3114 | - |
| 2.5442 | 11800 | 0.2759 | - |
| 2.5550 | 11850 | 0.3006 | - |
| 2.5658 | 11900 | 0.2647 | - |
| 2.5765 | 11950 | 0.225 | - |
| 2.5873 | 12000 | 0.2904 | - |
| 2.5981 | 12050 | 0.2776 | - |
| 2.6089 | 12100 | 0.3102 | - |
| 2.6197 | 12150 | 0.2499 | - |
| 2.6304 | 12200 | 0.2763 | - |
| 2.6412 | 12250 | 0.2645 | - |
| 2.6520 | 12300 | 0.3281 | - |
| 2.6628 | 12350 | 0.1793 | - |
| 2.6736 | 12400 | 0.3369 | - |
| 2.6843 | 12450 | 0.2598 | - |
| 2.6951 | 12500 | 0.3334 | - |
| 2.7059 | 12550 | 0.2935 | - |
| 2.7167 | 12600 | 0.4243 | - |
| 2.7275 | 12650 | 0.2212 | - |
| 2.7382 | 12700 | 0.2187 | - |
| 2.7490 | 12750 | 0.3004 | - |
| 2.7598 | 12800 | 0.4244 | - |
| 2.7706 | 12850 | 0.2242 | - |
| 2.7814 | 12900 | 0.3072 | - |
| 2.7922 | 12950 | 0.3468 | - |
| 2.8029 | 13000 | 0.2112 | - |
| 2.8137 | 13050 | 0.2935 | - |
| 2.8245 | 13100 | 0.2618 | - |
| 2.8353 | 13150 | 0.266 | - |
| 2.8461 | 13200 | 0.2458 | - |
| 2.8568 | 13250 | 0.2244 | - |
| 2.8676 | 13300 | 0.2764 | - |
| 2.8784 | 13350 | 0.2262 | - |
| 2.8892 | 13400 | 0.2232 | - |
| 2.9000 | 13450 | 0.2353 | - |
| 2.9107 | 13500 | 0.3661 | - |
| 2.9215 | 13550 | 0.1905 | - |
| 2.9323 | 13600 | 0.3493 | - |
| 2.9431 | 13650 | 0.2481 | - |
| 2.9539 | 13700 | 0.23 | - |
| 2.9646 | 13750 | 0.2407 | - |
| 2.9754 | 13800 | 0.2673 | - |
| 2.9862 | 13850 | 0.2091 | - |
| 2.9970 | 13900 | 0.2471 | - |
| 0.0004 | 1 | 0.287 | - |
| 0.0185 | 50 | 0.285 | - |
| 0.0370 | 100 | 0.233 | - |
| 0.0555 | 150 | 0.2874 | - |
| 0.0739 | 200 | 0.2599 | - |
| 0.0924 | 250 | 0.284 | - |
| 0.1109 | 300 | 0.3046 | - |
| 0.1294 | 350 | 0.2751 | - |
| 0.1479 | 400 | 0.2343 | - |
| 0.1664 | 450 | 0.2809 | - |
| 0.1848 | 500 | 0.2178 | - |
| 0.2033 | 550 | 0.2654 | - |
| 0.2218 | 600 | 0.2673 | - |
| 0.2403 | 650 | 0.2628 | - |
| 0.2588 | 700 | 0.279 | - |
| 0.2773 | 750 | 0.2448 | - |
| 0.2957 | 800 | 0.2502 | - |
| 0.3142 | 850 | 0.3343 | - |
| 0.3327 | 900 | 0.2669 | - |
| 0.3512 | 950 | 0.2714 | - |
| 0.3697 | 1000 | 0.3234 | - |
| 0.3882 | 1050 | 0.2892 | - |
| 0.4067 | 1100 | 0.2437 | - |
| 0.4251 | 1150 | 0.2409 | - |
| 0.4436 | 1200 | 0.2402 | - |
| 0.4621 | 1250 | 0.2479 | - |
| 0.4806 | 1300 | 0.2323 | - |
| 0.4991 | 1350 | 0.2474 | - |
| 0.5176 | 1400 | 0.319 | - |
| 0.5360 | 1450 | 0.3341 | - |
| 0.5545 | 1500 | 0.3162 | - |
| 0.5730 | 1550 | 0.2973 | - |
| 0.5915 | 1600 | 0.2975 | - |
| 0.6100 | 1650 | 0.2828 | - |
| 0.6285 | 1700 | 0.2625 | - |
| 0.6470 | 1750 | 0.2769 | - |
| 0.6654 | 1800 | 0.271 | - |
| 0.6839 | 1850 | 0.2538 | - |
| 0.7024 | 1900 | 0.1979 | - |
| 0.7209 | 1950 | 0.3117 | - |
| 0.7394 | 2000 | 0.2247 | - |
| 0.7579 | 2050 | 0.3248 | - |
| 0.7763 | 2100 | 0.2515 | - |
| 0.7948 | 2150 | 0.2877 | - |
| 0.8133 | 2200 | 0.3182 | - |
| 0.8318 | 2250 | 0.2772 | - |
| 0.8503 | 2300 | 0.2423 | - |
| 0.8688 | 2350 | 0.2638 | - |
| 0.8872 | 2400 | 0.226 | - |
| 0.9057 | 2450 | 0.306 | - |
| 0.9242 | 2500 | 0.2072 | - |
| 0.9427 | 2550 | 0.2434 | - |
| 0.9612 | 2600 | 0.2712 | - |
| 0.9797 | 2650 | 0.3225 | - |
| 0.9982 | 2700 | 0.2534 | - |
| 1.0166 | 2750 | 0.2364 | - |
| 1.0351 | 2800 | 0.241 | - |
| 1.0536 | 2850 | 0.2165 | - |
| 1.0721 | 2900 | 0.2719 | - |
| 1.0906 | 2950 | 0.2694 | - |
| 1.1091 | 3000 | 0.2562 | - |
| 1.1275 | 3050 | 0.2994 | - |
| 1.1460 | 3100 | 0.2477 | - |
| 1.1645 | 3150 | 0.231 | - |
| 1.1830 | 3200 | 0.2751 | - |
| 1.2015 | 3250 | 0.2543 | - |
| 1.2200 | 3300 | 0.2468 | - |
| 1.2384 | 3350 | 0.217 | - |
| 1.2569 | 3400 | 0.2664 | - |
| 1.2754 | 3450 | 0.2556 | - |
| 1.2939 | 3500 | 0.2334 | - |
| 1.3124 | 3550 | 0.2396 | - |
| 1.3309 | 3600 | 0.2383 | - |
| 1.3494 | 3650 | 0.2635 | - |
| 1.3678 | 3700 | 0.2652 | - |
| 1.3863 | 3750 | 0.2573 | - |
| 1.4048 | 3800 | 0.2211 | - |
| 1.4233 | 3850 | 0.2244 | - |
| 1.4418 | 3900 | 0.2399 | - |
| 1.4603 | 3950 | 0.2587 | - |
| 1.4787 | 4000 | 0.304 | - |
| 1.4972 | 4050 | 0.287 | - |
| 1.5157 | 4100 | 0.2667 | - |
| 1.5342 | 4150 | 0.3251 | - |
| 1.5527 | 4200 | 0.2641 | - |
| 1.5712 | 4250 | 0.2576 | - |
| 1.5896 | 4300 | 0.3057 | - |
| 1.6081 | 4350 | 0.2145 | - |
| 1.6266 | 4400 | 0.2665 | - |
| 1.6451 | 4450 | 0.2756 | - |
| 1.6636 | 4500 | 0.3089 | - |
| 1.6821 | 4550 | 0.3013 | - |
| 1.7006 | 4600 | 0.2337 | - |
| 1.7190 | 4650 | 0.2538 | - |
| 1.7375 | 4700 | 0.2428 | - |
| 1.7560 | 4750 | 0.2694 | - |
| 1.7745 | 4800 | 0.2367 | - |
| 1.7930 | 4850 | 0.2656 | - |
| 1.8115 | 4900 | 0.2405 | - |
| 1.8299 | 4950 | 0.2381 | - |
| 1.8484 | 5000 | 0.2363 | - |
| 1.8669 | 5050 | 0.2395 | - |
| 1.8854 | 5100 | 0.3183 | - |
| 1.9039 | 5150 | 0.2918 | - |
| 1.9224 | 5200 | 0.2985 | - |
| 1.9409 | 5250 | 0.3331 | - |
| 1.9593 | 5300 | 0.2716 | - |
| 1.9778 | 5350 | 0.2529 | - |
| 1.9963 | 5400 | 0.2557 | - |
| 2.0148 | 5450 | 0.2618 | - |
| 2.0333 | 5500 | 0.296 | - |
| 2.0518 | 5550 | 0.2866 | - |
| 2.0702 | 5600 | 0.2445 | - |
| 2.0887 | 5650 | 0.2464 | - |
| 2.1072 | 5700 | 0.2247 | - |
| 2.1257 | 5750 | 0.2906 | - |
| 2.1442 | 5800 | 0.2413 | - |
| 2.1627 | 5850 | 0.2805 | - |
| 2.1811 | 5900 | 0.2777 | - |
| 2.1996 | 5950 | 0.2151 | - |
| 2.2181 | 6000 | 0.2938 | - |
| 2.2366 | 6050 | 0.2569 | - |
| 2.2551 | 6100 | 0.2523 | - |
| 2.2736 | 6150 | 0.2649 | - |
| 2.2921 | 6200 | 0.2265 | - |
| 2.3105 | 6250 | 0.216 | - |
| 2.3290 | 6300 | 0.3309 | - |
| 2.3475 | 6350 | 0.2815 | - |
| 2.3660 | 6400 | 0.2566 | - |
| 2.3845 | 6450 | 0.237 | - |
| 2.4030 | 6500 | 0.2165 | - |
| 2.4214 | 6550 | 0.2975 | - |
| 2.4399 | 6600 | 0.2402 | - |
| 2.4584 | 6650 | 0.2943 | - |
| 2.4769 | 6700 | 0.2522 | - |
| 2.4954 | 6750 | 0.2473 | - |
| 2.5139 | 6800 | 0.2652 | - |
| 2.5323 | 6850 | 0.244 | - |
| 2.5508 | 6900 | 0.2488 | - |
| 2.5693 | 6950 | 0.2726 | - |
| 2.5878 | 7000 | 0.2282 | - |
| 2.6063 | 7050 | 0.2386 | - |
| 2.6248 | 7100 | 0.3269 | - |
| 2.6433 | 7150 | 0.2401 | - |
| 2.6617 | 7200 | 0.284 | - |
| 2.6802 | 7250 | 0.3263 | - |
| 2.6987 | 7300 | 0.3019 | - |
| 2.7172 | 7350 | 0.2364 | - |
| 2.7357 | 7400 | 0.2219 | - |
| 2.7542 | 7450 | 0.2798 | - |
| 2.7726 | 7500 | 0.2605 | - |
| 2.7911 | 7550 | 0.2958 | - |
| 2.8096 | 7600 | 0.2028 | - |
| 2.8281 | 7650 | 0.2577 | - |
| 2.8466 | 7700 | 0.2686 | - |
| 2.8651 | 7750 | 0.2894 | - |
| 2.8835 | 7800 | 0.3136 | - |
| 2.9020 | 7850 | 0.2417 | - |
| 2.9205 | 7900 | 0.276 | - |
| 2.9390 | 7950 | 0.2608 | - |
| 2.9575 | 8000 | 0.2545 | - |
| 2.9760 | 8050 | 0.2539 | - |
| 2.9945 | 8100 | 0.1995 | - |
### Framework Versions
- Python: 3.9.16
- SetFit: 1.0.1
- Sentence Transformers: 2.2.2
- Transformers: 4.35.0
- PyTorch: 2.1.0+cu121
- Datasets: 2.14.6
- Tokenizers: 0.14.1
## Citation
### BibTeX
```bibtex
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
```