mann2107's picture
Update README.md
cda558e verified
metadata
base_model: sentence-transformers/all-MiniLM-L6-v2
library_name: setfit
metrics:
  - accuracy
pipeline_tag: text-classification
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
widget:
  - text: Thank you for your email. Please go ahead and issue. Please invoice in KES
  - text: >-
      Hi, We are missing some invoices, can you please provide it. 02 - 12 -
      2020 AGENT FEE 8900784339018 $21.00 02 - 19 - 2020 AGENT FEE 0017417554160
      $22.00 02 - 19 - 2020 AGENT FEE 0017417554143 $22.00 02 - 19 - 2020 AGENT
      FEE 8900783383420 $21.00
  - text: >-
      We need your assistance with the payment for the recent office supplies
      order. Let us know once it's done.
  - text: >-
      I have reported this in November and not only was the trip supposed to be
      cancelled and credited I was double billed and the billing has not been
      corrected. The total credit should be $667.20. Please confirm this will be
      done.
  - text: >-
      The invoice for the travel arrangements needs to be settled. Kindly
      provide payment confirmation.
inference: true

SetFit with sentence-transformers/all-MiniLM-L6-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/all-MiniLM-L6-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("mann2107/BCMPIIRAB_MiniLM_ALLNewV2")
# Run inference
preds = model("Thank you for your email. Please go ahead and issue. Please invoice in KES")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 1 25.6577 136
Label Training Sample Count
0 24
1 24
2 24
3 24
4 24
5 24
6 24
7 24
8 24
9 24
10 24
11 24
12 24
13 24

Training Hyperparameters

  • batch_size: (8, 8)
  • num_epochs: (5, 5)
  • max_steps: -1
  • sampling_strategy: oversampling
  • num_iterations: 68
  • body_learning_rate: (1.44030579311381e-05, 1.44030579311381e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • max_length: 512
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: True

Training Results

Epoch Step Training Loss Validation Loss
0.0002 1 0.2917 -
0.0088 50 0.2434 -
0.0175 100 0.2053 -
0.0263 150 0.1789 -
0.0350 200 0.2249 -
0.0438 250 0.1773 -
0.0525 300 0.1648 -
0.0613 350 0.2617 -
0.0700 400 0.1342 -
0.0788 450 0.1064 -
0.0875 500 0.1273 -
0.0963 550 0.1248 -
0.1050 600 0.2013 -
0.1138 650 0.1979 -
0.1225 700 0.1631 -
0.1313 750 0.1079 -
0.1401 800 0.0858 -
0.1488 850 0.0999 -
0.1576 900 0.0638 -
0.1663 950 0.1287 -
0.1751 1000 0.1408 -
0.1838 1050 0.1902 -
0.1926 1100 0.0648 -
0.2013 1150 0.1383 -
0.2101 1200 0.0609 -
0.2188 1250 0.0865 -
0.2276 1300 0.1069 -
0.2363 1350 0.051 -
0.2451 1400 0.0692 -
0.2539 1450 0.123 -
0.2626 1500 0.0758 -
0.2714 1550 0.0835 -
0.2801 1600 0.0523 -
0.2889 1650 0.0946 -
0.2976 1700 0.0445 -
0.3064 1750 0.0248 -
0.3151 1800 0.0373 -
0.3239 1850 0.0248 -
0.3326 1900 0.0446 -
0.3414 1950 0.0142 -
0.3501 2000 0.023 -
0.3589 2050 0.0119 -
0.3676 2100 0.0383 -
0.3764 2150 0.0188 -
0.3852 2200 0.0204 -
0.3939 2250 0.0109 -
0.4027 2300 0.0273 -
0.4114 2350 0.0216 -
0.4202 2400 0.0073 -
0.4289 2450 0.0338 -
0.4377 2500 0.0047 -
0.4464 2550 0.0096 -
0.4552 2600 0.0069 -
0.4639 2650 0.0078 -
0.4727 2700 0.0122 -
0.4814 2750 0.0578 -
0.4902 2800 0.0074 -
0.4989 2850 0.0103 -
0.5077 2900 0.0092 -
0.5165 2950 0.004 -
0.5252 3000 0.0061 -
0.5340 3050 0.0214 -
0.5427 3100 0.0048 -
0.5515 3150 0.0036 -
0.5602 3200 0.0041 -
0.5690 3250 0.0151 -
0.5777 3300 0.0042 -
0.5865 3350 0.0029 -
0.5952 3400 0.0021 -
0.6040 3450 0.0018 -
0.6127 3500 0.0058 -
0.6215 3550 0.0011 -
0.6303 3600 0.0078 -
0.6390 3650 0.0011 -
0.6478 3700 0.0017 -
0.6565 3750 0.0022 -
0.6653 3800 0.0016 -
0.6740 3850 0.002 -
0.6828 3900 0.0023 -
0.6915 3950 0.0011 -
0.7003 4000 0.0012 -
0.7090 4050 0.0007 -
0.7178 4100 0.0021 -
0.7265 4150 0.0019 -
0.7353 4200 0.002 -
0.7440 4250 0.0018 -
0.7528 4300 0.0029 -
0.7616 4350 0.0015 -
0.7703 4400 0.0022 -
0.7791 4450 0.0012 -
0.7878 4500 0.0007 -
0.7966 4550 0.0015 -
0.8053 4600 0.0011 -
0.8141 4650 0.0016 -
0.8228 4700 0.0009 -
0.8316 4750 0.0007 -
0.8403 4800 0.0011 -
0.8491 4850 0.001 -
0.8578 4900 0.0008 -
0.8666 4950 0.0014 -
0.8754 5000 0.0022 -
0.8841 5050 0.0012 -
0.8929 5100 0.0007 -
0.9016 5150 0.0014 -
0.9104 5200 0.0007 -
0.9191 5250 0.0012 -
0.9279 5300 0.0011 -
0.9366 5350 0.0012 -
0.9454 5400 0.0029 -
0.9541 5450 0.001 -
0.9629 5500 0.0011 -
0.9716 5550 0.0004 -
0.9804 5600 0.0009 -
0.9891 5650 0.0004 -
0.9979 5700 0.003 -
1.0 5712 - 0.0459
1.0067 5750 0.0014 -
1.0154 5800 0.0008 -
1.0242 5850 0.0009 -
1.0329 5900 0.0007 -
1.0417 5950 0.0007 -
1.0504 6000 0.0006 -
1.0592 6050 0.0008 -
1.0679 6100 0.0006 -
1.0767 6150 0.0006 -
1.0854 6200 0.0007 -
1.0942 6250 0.0025 -
1.1029 6300 0.0006 -
1.1117 6350 0.0009 -
1.1204 6400 0.0009 -
1.1292 6450 0.0009 -
1.1380 6500 0.0006 -
1.1467 6550 0.0004 -
1.1555 6600 0.0014 -
1.1642 6650 0.0029 -
1.1730 6700 0.0004 -
1.1817 6750 0.0027 -
1.1905 6800 0.0003 -
1.1992 6850 0.0003 -
1.2080 6900 0.0006 -
1.2167 6950 0.0015 -
1.2255 7000 0.0005 -
1.2342 7050 0.0005 -
1.2430 7100 0.0016 -
1.2518 7150 0.0005 -
1.2605 7200 0.0003 -
1.2693 7250 0.0006 -
1.2780 7300 0.0007 -
1.2868 7350 0.0004 -
1.2955 7400 0.0007 -
1.3043 7450 0.0007 -
1.3130 7500 0.0007 -
1.3218 7550 0.0003 -
1.3305 7600 0.0002 -
1.3393 7650 0.0002 -
1.3480 7700 0.0005 -
1.3568 7750 0.0014 -
1.3655 7800 0.0012 -
1.3743 7850 0.0002 -
1.3831 7900 0.0002 -
1.3918 7950 0.0003 -
1.4006 8000 0.0005 -
1.4093 8050 0.0006 -
1.4181 8100 0.0003 -
1.4268 8150 0.0009 -
1.4356 8200 0.0004 -
1.4443 8250 0.0002 -
1.4531 8300 0.0004 -
1.4618 8350 0.0008 -
1.4706 8400 0.0002 -
1.4793 8450 0.0004 -
1.4881 8500 0.0006 -
1.4968 8550 0.0011 -
1.5056 8600 0.0003 -
1.5144 8650 0.0003 -
1.5231 8700 0.0004 -
1.5319 8750 0.0004 -
1.5406 8800 0.0002 -
1.5494 8850 0.0007 -
1.5581 8900 0.0003 -
1.5669 8950 0.0002 -
1.5756 9000 0.0007 -
1.5844 9050 0.0005 -
1.5931 9100 0.0005 -
1.6019 9150 0.0011 -
1.6106 9200 0.0004 -
1.6194 9250 0.0004 -
1.6282 9300 0.0003 -
1.6369 9350 0.0002 -
1.6457 9400 0.0003 -
1.6544 9450 0.0006 -
1.6632 9500 0.0004 -
1.6719 9550 0.0004 -
1.6807 9600 0.0006 -
1.6894 9650 0.0001 -
1.6982 9700 0.0002 -
1.7069 9750 0.0004 -
1.7157 9800 0.0004 -
1.7244 9850 0.0001 -
1.7332 9900 0.0004 -
1.7419 9950 0.0004 -
1.7507 10000 0.0006 -
1.7595 10050 0.0003 -
1.7682 10100 0.0002 -
1.7770 10150 0.0004 -
1.7857 10200 0.0004 -
1.7945 10250 0.0002 -
1.8032 10300 0.0008 -
1.8120 10350 0.0004 -
1.8207 10400 0.0005 -
1.8295 10450 0.0004 -
1.8382 10500 0.0001 -
1.8470 10550 0.0003 -
1.8557 10600 0.0003 -
1.8645 10650 0.0005 -
1.8732 10700 0.0005 -
1.8820 10750 0.0003 -
1.8908 10800 0.0001 -
1.8995 10850 0.0002 -
1.9083 10900 0.0001 -
1.9170 10950 0.0003 -
1.9258 11000 0.0005 -
1.9345 11050 0.0003 -
1.9433 11100 0.0004 -
1.9520 11150 0.0007 -
1.9608 11200 0.0002 -
1.9695 11250 0.0003 -
1.9783 11300 0.0001 -
1.9870 11350 0.0001 -
1.9958 11400 0.0002 -
2.0 11424 - 0.042
2.0046 11450 0.0003 -
2.0133 11500 0.0002 -
2.0221 11550 0.0002 -
2.0308 11600 0.0002 -
2.0396 11650 0.0003 -
2.0483 11700 0.0003 -
2.0571 11750 0.0002 -
2.0658 11800 0.0002 -
2.0746 11850 0.0002 -
2.0833 11900 0.0002 -
2.0921 11950 0.0001 -
2.1008 12000 0.0003 -
2.1096 12050 0.0005 -
2.1183 12100 0.0002 -
2.1271 12150 0.0003 -
2.1359 12200 0.0002 -
2.1446 12250 0.0003 -
2.1534 12300 0.0003 -
2.1621 12350 0.0001 -
2.1709 12400 0.0002 -
2.1796 12450 0.0002 -
2.1884 12500 0.0002 -
2.1971 12550 0.0002 -
2.2059 12600 0.0001 -
2.2146 12650 0.0002 -
2.2234 12700 0.0003 -
2.2321 12750 0.0003 -
2.2409 12800 0.0004 -
2.2496 12850 0.0002 -
2.2584 12900 0.0002 -
2.2672 12950 0.0003 -
2.2759 13000 0.0002 -
2.2847 13050 0.0002 -
2.2934 13100 0.0002 -
2.3022 13150 0.0001 -
2.3109 13200 0.0002 -
2.3197 13250 0.0001 -
2.3284 13300 0.0002 -
2.3372 13350 0.0003 -
2.3459 13400 0.0002 -
2.3547 13450 0.0001 -
2.3634 13500 0.0002 -
2.3722 13550 0.0001 -
2.3810 13600 0.0006 -
2.3897 13650 0.0001 -
2.3985 13700 0.0002 -
2.4072 13750 0.0002 -
2.4160 13800 0.0004 -
2.4247 13850 0.0001 -
2.4335 13900 0.0003 -
2.4422 13950 0.0001 -
2.4510 14000 0.0001 -
2.4597 14050 0.0001 -
2.4685 14100 0.0005 -
2.4772 14150 0.0002 -
2.4860 14200 0.0001 -
2.4947 14250 0.0003 -
2.5035 14300 0.0005 -
2.5123 14350 0.0002 -
2.5210 14400 0.0002 -
2.5298 14450 0.0003 -
2.5385 14500 0.0001 -
2.5473 14550 0.0001 -
2.5560 14600 0.0002 -
2.5648 14650 0.0002 -
2.5735 14700 0.0001 -
2.5823 14750 0.0001 -
2.5910 14800 0.0001 -
2.5998 14850 0.0003 -
2.6085 14900 0.0002 -
2.6173 14950 0.0001 -
2.6261 15000 0.0001 -
2.6348 15050 0.0001 -
2.6436 15100 0.0001 -
2.6523 15150 0.0002 -
2.6611 15200 0.0001 -
2.6698 15250 0.0002 -
2.6786 15300 0.0002 -
2.6873 15350 0.0002 -
2.6961 15400 0.0002 -
2.7048 15450 0.0002 -
2.7136 15500 0.0001 -
2.7223 15550 0.0002 -
2.7311 15600 0.0002 -
2.7398 15650 0.0003 -
2.7486 15700 0.0002 -
2.7574 15750 0.0001 -
2.7661 15800 0.0002 -
2.7749 15850 0.0002 -
2.7836 15900 0.0003 -
2.7924 15950 0.0004 -
2.8011 16000 0.0007 -
2.8099 16050 0.0001 -
2.8186 16100 0.0001 -
2.8274 16150 0.0002 -
2.8361 16200 0.0002 -
2.8449 16250 0.0001 -
2.8536 16300 0.0001 -
2.8624 16350 0.0002 -
2.8711 16400 0.0002 -
2.8799 16450 0.0001 -
2.8887 16500 0.0002 -
2.8974 16550 0.0002 -
2.9062 16600 0.0001 -
2.9149 16650 0.0001 -
2.9237 16700 0.0001 -
2.9324 16750 0.0003 -
2.9412 16800 0.0002 -
2.9499 16850 0.0003 -
2.9587 16900 0.0001 -
2.9674 16950 0.0002 -
2.9762 17000 0.0001 -
2.9849 17050 0.0001 -
2.9937 17100 0.0001 -
3.0 17136 - 0.0419
3.0025 17150 0.0002 -
3.0112 17200 0.0002 -
3.0200 17250 0.0003 -
3.0287 17300 0.0001 -
3.0375 17350 0.0002 -
3.0462 17400 0.0001 -
3.0550 17450 0.0002 -
3.0637 17500 0.0002 -
3.0725 17550 0.0002 -
3.0812 17600 0.0001 -
3.0900 17650 0.0001 -
3.0987 17700 0.0001 -
3.1075 17750 0.0001 -
3.1162 17800 0.0001 -
3.125 17850 0.0001 -
3.1338 17900 0.0002 -
3.1425 17950 0.0001 -
3.1513 18000 0.0003 -
3.1600 18050 0.0001 -
3.1688 18100 0.0001 -
3.1775 18150 0.0001 -
3.1863 18200 0.0002 -
3.1950 18250 0.0002 -
3.2038 18300 0.0001 -
3.2125 18350 0.0001 -
3.2213 18400 0.0001 -
3.2300 18450 0.0002 -
3.2388 18500 0.0001 -
3.2475 18550 0.0002 -
3.2563 18600 0.0001 -
3.2651 18650 0.0002 -
3.2738 18700 0.0001 -
3.2826 18750 0.0001 -
3.2913 18800 0.0001 -
3.3001 18850 0.0001 -
3.3088 18900 0.0003 -
3.3176 18950 0.0002 -
3.3263 19000 0.0001 -
3.3351 19050 0.0003 -
3.3438 19100 0.0001 -
3.3526 19150 0.0001 -
3.3613 19200 0.0001 -
3.3701 19250 0.0001 -
3.3789 19300 0.0001 -
3.3876 19350 0.0002 -
3.3964 19400 0.0001 -
3.4051 19450 0.0001 -
3.4139 19500 0.0001 -
3.4226 19550 0.0001 -
3.4314 19600 0.0001 -
3.4401 19650 0.0001 -
3.4489 19700 0.0002 -
3.4576 19750 0.0001 -
3.4664 19800 0.0001 -
3.4751 19850 0.0001 -
3.4839 19900 0.0001 -
3.4926 19950 0.0001 -
3.5014 20000 0.0001 -
3.5102 20050 0.0002 -
3.5189 20100 0.0003 -
3.5277 20150 0.0001 -
3.5364 20200 0.0002 -
3.5452 20250 0.0001 -
3.5539 20300 0.0001 -
3.5627 20350 0.0001 -
3.5714 20400 0.0004 -
3.5802 20450 0.0001 -
3.5889 20500 0.0001 -
3.5977 20550 0.0001 -
3.6064 20600 0.0002 -
3.6152 20650 0.0001 -
3.6239 20700 0.0001 -
3.6327 20750 0.0 -
3.6415 20800 0.0002 -
3.6502 20850 0.0001 -
3.6590 20900 0.0001 -
3.6677 20950 0.0002 -
3.6765 21000 0.0001 -
3.6852 21050 0.0001 -
3.6940 21100 0.0001 -
3.7027 21150 0.0002 -
3.7115 21200 0.0004 -
3.7202 21250 0.0001 -
3.7290 21300 0.0002 -
3.7377 21350 0.0001 -
3.7465 21400 0.0004 -
3.7553 21450 0.0002 -
3.7640 21500 0.0001 -
3.7728 21550 0.0001 -
3.7815 21600 0.0001 -
3.7903 21650 0.0001 -
3.7990 21700 0.0001 -
3.8078 21750 0.0001 -
3.8165 21800 0.0 -
3.8253 21850 0.0 -
3.8340 21900 0.0001 -
3.8428 21950 0.0003 -
3.8515 22000 0.0001 -
3.8603 22050 0.0001 -
3.8690 22100 0.0002 -
3.8778 22150 0.0001 -
3.8866 22200 0.0003 -
3.8953 22250 0.0001 -
3.9041 22300 0.0 -
3.9128 22350 0.0001 -
3.9216 22400 0.0002 -
3.9303 22450 0.0001 -
3.9391 22500 0.0001 -
3.9478 22550 0.0 -
3.9566 22600 0.0003 -
3.9653 22650 0.0001 -
3.9741 22700 0.0001 -
3.9828 22750 0.0001 -
3.9916 22800 0.0002 -
4.0 22848 - 0.0419
4.0004 22850 0.0 -
4.0091 22900 0.0001 -
4.0179 22950 0.0001 -
4.0266 23000 0.0001 -
4.0354 23050 0.0001 -
4.0441 23100 0.0002 -
4.0529 23150 0.0001 -
4.0616 23200 0.0001 -
4.0704 23250 0.0002 -
4.0791 23300 0.0 -
4.0879 23350 0.0001 -
4.0966 23400 0.0001 -
4.1054 23450 0.0001 -
4.1141 23500 0.0001 -
4.1229 23550 0.0002 -
4.1317 23600 0.0001 -
4.1404 23650 0.0001 -
4.1492 23700 0.0001 -
4.1579 23750 0.0002 -
4.1667 23800 0.0002 -
4.1754 23850 0.0001 -
4.1842 23900 0.0001 -
4.1929 23950 0.0001 -
4.2017 24000 0.0001 -
4.2104 24050 0.0001 -
4.2192 24100 0.0001 -
4.2279 24150 0.0 -
4.2367 24200 0.0001 -
4.2454 24250 0.0001 -
4.2542 24300 0.0003 -
4.2630 24350 0.0 -
4.2717 24400 0.0001 -
4.2805 24450 0.0 -
4.2892 24500 0.0001 -
4.2980 24550 0.0001 -
4.3067 24600 0.0002 -
4.3155 24650 0.0 -
4.3242 24700 0.0001 -
4.3330 24750 0.0001 -
4.3417 24800 0.0001 -
4.3505 24850 0.0001 -
4.3592 24900 0.0001 -
4.3680 24950 0.0 -
4.3768 25000 0.0002 -
4.3855 25050 0.0001 -
4.3943 25100 0.0001 -
4.4030 25150 0.0001 -
4.4118 25200 0.0001 -
4.4205 25250 0.0001 -
4.4293 25300 0.0002 -
4.4380 25350 0.0002 -
4.4468 25400 0.0001 -
4.4555 25450 0.0001 -
4.4643 25500 0.0001 -
4.4730 25550 0.0001 -
4.4818 25600 0.0001 -
4.4905 25650 0.0001 -
4.4993 25700 0.0001 -
4.5081 25750 0.0001 -
4.5168 25800 0.0001 -
4.5256 25850 0.0001 -
4.5343 25900 0.0001 -
4.5431 25950 0.0001 -
4.5518 26000 0.0 -
4.5606 26050 0.0 -
4.5693 26100 0.0001 -
4.5781 26150 0.0001 -
4.5868 26200 0.0001 -
4.5956 26250 0.0001 -
4.6043 26300 0.0001 -
4.6131 26350 0.0001 -
4.6218 26400 0.0002 -
4.6306 26450 0.0001 -
4.6394 26500 0.0001 -
4.6481 26550 0.0001 -
4.6569 26600 0.0001 -
4.6656 26650 0.0 -
4.6744 26700 0.0002 -
4.6831 26750 0.0 -
4.6919 26800 0.0001 -
4.7006 26850 0.0002 -
4.7094 26900 0.0002 -
4.7181 26950 0.0001 -
4.7269 27000 0.0001 -
4.7356 27050 0.0001 -
4.7444 27100 0.0 -
4.7532 27150 0.0001 -
4.7619 27200 0.0001 -
4.7707 27250 0.0001 -
4.7794 27300 0.0 -
4.7882 27350 0.0001 -
4.7969 27400 0.0001 -
4.8057 27450 0.0002 -
4.8144 27500 0.0 -
4.8232 27550 0.0001 -
4.8319 27600 0.0001 -
4.8407 27650 0.0001 -
4.8494 27700 0.0 -
4.8582 27750 0.0001 -
4.8669 27800 0.0001 -
4.8757 27850 0.0001 -
4.8845 27900 0.0001 -
4.8932 27950 0.0001 -
4.9020 28000 0.0001 -
4.9107 28050 0.0001 -
4.9195 28100 0.0 -
4.9282 28150 0.0001 -
4.9370 28200 0.0001 -
4.9457 28250 0.0001 -
4.9545 28300 0.0001 -
4.9632 28350 0.0001 -
4.9720 28400 0.0001 -
4.9807 28450 0.0001 -
4.9895 28500 0.0002 -
4.9982 28550 0.0 -
5.0 28560 - 0.0425
  • The bold row denotes the saved checkpoint.

Framework Versions

  • Python: 3.10.12
  • SetFit: 1.1.0.dev0
  • Sentence Transformers: 3.0.1
  • Transformers: 4.42.4
  • PyTorch: 2.3.1+cu121
  • Datasets: 2.20.0
  • Tokenizers: 0.19.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}