--- base_model: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 library_name: setfit metrics: - accuracy pipeline_tag: text-classification tags: - setfit - sentence-transformers - text-classification - generated_from_setfit_trainer widget: - text: Bei den Koalitionsverhandlungen von SPD, Grünen und FDP war die Einführung eines generellen Tempolimits auf deutschen Autobahnen am Widerstand der Liberalen gescheitert. Auch bei einem vor kurzem von den Koalitionsspitzen beschlossenen Maßnahmenpaket auch zum Energiesparen fehlte ein Tempolimit. - text: 'Deutschland will 2045 klimaneutral sein. Bis dahin müssen die Emissionen nach und nach sinken. Das bedeutet, dass alle Wirtschafts- und Lebensbereiche sich von der Nutzung fossiler Energien verabschieden müssen – so auch das Heizen. Statt mit Öl und Gas müssen die Gebäude also mit erneuerbaren Optionen aufgewärmt werden, zum Beispiel mit Wärmepumpen, Solar- oder Geothermie. Bislang geht es dabei aber kaum voran: Noch im ersten Quartal dieses Jahres waren laut des Bundesverbands der Deutschen Heizungsindustrie mehr als die Hälfte der verkauften Heizungen gasbetrieben. Ganz grundsätzlich sieht das neue Heizungsgesetz nun vor, dass neue Heizungen ab dem kommenden Jahr mindestens zu 65 Prozent erneuerbar betrieben werden. Durch Ausnahmen wie die bei wasserstofftauglichen Gasheizungen soll das aber nur noch eingeschränkt gelten.' - text: Clemens Traub bezeichnete FFF als Bewegung, in der Arzttöchter anderen die Welt erklären. Wie wollen Sie denn die Männer von der Autobahnmeisterer oder die Fernpendlerin erreichen?Niemand schlägt vor, dass in Deutschland alle Autobahnen rückgebaut werden sollen. Natürlich müssen marode Straßen und Brücken saniert werden, damit sich Menschen sicher bewegen können. Gleichzeitig sollte Mobilität so gestaltet werden, dass wir nicht durch jeden Weg, den wir zurücklegen, Klimaschäden produzieren, die sich nicht mehr auffangen lassen. - text: ', die Jugendvertretung Bayern der Gewerkschaft Nahrung Genussmittel Gaststätten NGG, die Bund-Naturschutz-Jugend, die Falken im Bezirk Südbayern, die Münchner Mieterschutzinitiative ›Ausspekuliert›, ein bundesweites Bündnis Armutsbetroffener ichbinarmutsbetroffen, FFF, das Bündnis Attac, der Paritätische Wohlfahrtsverband Bayern und der Sozialverband VdK Bayern.' - text: 'Am späten Vormittag zogen die Klima-Chaoten eine erste Zwischenbilanz:.Aimée Vanbaalen, Sprecherin der ›DLG›, über die Störungen: ›Unsere höchsten Erwartungen wurden deutlich übertroffen! An 27 Verkehrsknotenpunkten in Berlin kam es heute zu Protesten, drei Mal so viele wie noch im letzten Herbst.›' inference: true model-index: - name: SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 results: - task: type: text-classification name: Text Classification dataset: name: Unknown type: unknown split: test metrics: - type: accuracy value: 0.6916666666666667 name: Accuracy --- # SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. The model has been trained using an efficient few-shot learning technique that involves: 1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning. 2. Training a classification head with features from the fine-tuned Sentence Transformer. ## Model Details ### Model Description - **Model Type:** SetFit - **Sentence Transformer body:** [sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance - **Maximum Sequence Length:** 128 tokens - **Number of Classes:** 3 classes ### Model Sources - **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit) - **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055) - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit) ### Model Labels | Label | Examples | |:-----------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | OPPOSED | | | NEUTRAL | | | SUPPORTIVE | | ## Evaluation ### Metrics | Label | Accuracy | |:--------|:---------| | **all** | 0.6917 | ## Uses ### Direct Use for Inference First install the SetFit library: ```bash pip install setfit ``` Then you can load this model and run inference. ```python from setfit import SetFitModel # Download from the 🤗 Hub model = SetFitModel.from_pretrained("cbpuschmann/MiniLM-klimacoder_v0.1") # Run inference preds = model("Bei den Koalitionsverhandlungen von SPD, Grünen und FDP war die Einführung eines generellen Tempolimits auf deutschen Autobahnen am Widerstand der Liberalen gescheitert. Auch bei einem vor kurzem von den Koalitionsspitzen beschlossenen Maßnahmenpaket auch zum Energiesparen fehlte ein Tempolimit.") ``` ## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:-------------|:----|:--------|:----| | Word count | 15 | 65.3896 | 237 | | Label | Training Sample Count | |:-----------|:----------------------| | NEUTRAL | 219 | | OPPOSED | 125 | | SUPPORTIVE | 136 | ### Training Hyperparameters - batch_size: (128, 128) - num_epochs: (10, 10) - max_steps: -1 - sampling_strategy: oversampling - body_learning_rate: (2e-05, 1e-05) - head_learning_rate: 0.01 - loss: CosineSimilarityLoss - distance_metric: cosine_distance - margin: 0.25 - end_to_end: False - use_amp: False - warmup_proportion: 0.1 - l2_weight: 0.01 - seed: 42 - eval_max_steps: -1 - load_best_model_at_end: False ### Training Results | Epoch | Step | Training Loss | Validation Loss | |:------:|:-----:|:-------------:|:---------------:| | 0.0009 | 1 | 0.2764 | - | | 0.0431 | 50 | 0.2927 | - | | 0.0863 | 100 | 0.2729 | - | | 0.1294 | 150 | 0.2637 | - | | 0.1726 | 200 | 0.2562 | - | | 0.2157 | 250 | 0.2485 | - | | 0.2588 | 300 | 0.2386 | - | | 0.3020 | 350 | 0.22 | - | | 0.3451 | 400 | 0.1755 | - | | 0.3883 | 450 | 0.1235 | - | | 0.4314 | 500 | 0.073 | - | | 0.4745 | 550 | 0.0301 | - | | 0.5177 | 600 | 0.0105 | - | | 0.5608 | 650 | 0.0058 | - | | 0.6040 | 700 | 0.0049 | - | | 0.6471 | 750 | 0.0035 | - | | 0.6903 | 800 | 0.0031 | - | | 0.7334 | 850 | 0.0027 | - | | 0.7765 | 900 | 0.0027 | - | | 0.8197 | 950 | 0.0021 | - | | 0.8628 | 1000 | 0.0022 | - | | 0.9060 | 1050 | 0.0014 | - | | 0.9491 | 1100 | 0.0022 | - | | 0.9922 | 1150 | 0.0018 | - | | 1.0354 | 1200 | 0.0019 | - | | 1.0785 | 1250 | 0.0024 | - | | 1.1217 | 1300 | 0.0015 | - | | 1.1648 | 1350 | 0.0021 | - | | 1.2079 | 1400 | 0.0022 | - | | 1.2511 | 1450 | 0.0016 | - | | 1.2942 | 1500 | 0.0021 | - | | 1.3374 | 1550 | 0.0023 | - | | 1.3805 | 1600 | 0.0022 | - | | 1.4236 | 1650 | 0.0013 | - | | 1.4668 | 1700 | 0.0019 | - | | 1.5099 | 1750 | 0.0023 | - | | 1.5531 | 1800 | 0.0016 | - | | 1.5962 | 1850 | 0.0018 | - | | 1.6393 | 1900 | 0.0013 | - | | 1.6825 | 1950 | 0.0014 | - | | 1.7256 | 2000 | 0.0017 | - | | 1.7688 | 2050 | 0.0016 | - | | 1.8119 | 2100 | 0.0016 | - | | 1.8550 | 2150 | 0.0016 | - | | 1.8982 | 2200 | 0.0024 | - | | 1.9413 | 2250 | 0.0013 | - | | 1.9845 | 2300 | 0.0019 | - | | 2.0276 | 2350 | 0.0014 | - | | 2.0708 | 2400 | 0.0019 | - | | 2.1139 | 2450 | 0.0016 | - | | 2.1570 | 2500 | 0.002 | - | | 2.2002 | 2550 | 0.0011 | - | | 2.2433 | 2600 | 0.0014 | - | | 2.2865 | 2650 | 0.0016 | - | | 2.3296 | 2700 | 0.0013 | - | | 2.3727 | 2750 | 0.0013 | - | | 2.4159 | 2800 | 0.0022 | - | | 2.4590 | 2850 | 0.0017 | - | | 2.5022 | 2900 | 0.0016 | - | | 2.5453 | 2950 | 0.0015 | - | | 2.5884 | 3000 | 0.0021 | - | | 2.6316 | 3050 | 0.0022 | - | | 2.6747 | 3100 | 0.0019 | - | | 2.7179 | 3150 | 0.0014 | - | | 2.7610 | 3200 | 0.0013 | - | | 2.8041 | 3250 | 0.0012 | - | | 2.8473 | 3300 | 0.0014 | - | | 2.8904 | 3350 | 0.0023 | - | | 2.9336 | 3400 | 0.0018 | - | | 2.9767 | 3450 | 0.0017 | - | | 3.0198 | 3500 | 0.002 | - | | 3.0630 | 3550 | 0.0021 | - | | 3.1061 | 3600 | 0.0024 | - | | 3.1493 | 3650 | 0.0021 | - | | 3.1924 | 3700 | 0.0015 | - | | 3.2355 | 3750 | 0.0015 | - | | 3.2787 | 3800 | 0.0016 | - | | 3.3218 | 3850 | 0.0012 | - | | 3.3650 | 3900 | 0.0016 | - | | 3.4081 | 3950 | 0.0011 | - | | 3.4513 | 4000 | 0.0017 | - | | 3.4944 | 4050 | 0.0018 | - | | 3.5375 | 4100 | 0.0015 | - | | 3.5807 | 4150 | 0.0019 | - | | 3.6238 | 4200 | 0.0017 | - | | 3.6670 | 4250 | 0.0019 | - | | 3.7101 | 4300 | 0.0014 | - | | 3.7532 | 4350 | 0.0017 | - | | 3.7964 | 4400 | 0.0014 | - | | 3.8395 | 4450 | 0.0013 | - | | 3.8827 | 4500 | 0.002 | - | | 3.9258 | 4550 | 0.0014 | - | | 3.9689 | 4600 | 0.0021 | - | | 4.0121 | 4650 | 0.0017 | - | | 4.0552 | 4700 | 0.0018 | - | | 4.0984 | 4750 | 0.0012 | - | | 4.1415 | 4800 | 0.0017 | - | | 4.1846 | 4850 | 0.0022 | - | | 4.2278 | 4900 | 0.0012 | - | | 4.2709 | 4950 | 0.0014 | - | | 4.3141 | 5000 | 0.0016 | - | | 4.3572 | 5050 | 0.0016 | - | | 4.4003 | 5100 | 0.0015 | - | | 4.4435 | 5150 | 0.0015 | - | | 4.4866 | 5200 | 0.001 | - | | 4.5298 | 5250 | 0.0019 | - | | 4.5729 | 5300 | 0.0028 | - | | 4.6160 | 5350 | 0.0016 | - | | 4.6592 | 5400 | 0.0013 | - | | 4.7023 | 5450 | 0.0017 | - | | 4.7455 | 5500 | 0.0019 | - | | 4.7886 | 5550 | 0.0015 | - | | 4.8318 | 5600 | 0.002 | - | | 4.8749 | 5650 | 0.002 | - | | 4.9180 | 5700 | 0.0023 | - | | 4.9612 | 5750 | 0.0012 | - | | 5.0043 | 5800 | 0.0012 | - | | 5.0475 | 5850 | 0.0016 | - | | 5.0906 | 5900 | 0.0014 | - | | 5.1337 | 5950 | 0.0011 | - | | 5.1769 | 6000 | 0.0017 | - | | 5.2200 | 6050 | 0.0015 | - | | 5.2632 | 6100 | 0.0022 | - | | 5.3063 | 6150 | 0.0012 | - | | 5.3494 | 6200 | 0.0018 | - | | 5.3926 | 6250 | 0.0015 | - | | 5.4357 | 6300 | 0.002 | - | | 5.4789 | 6350 | 0.0017 | - | | 5.5220 | 6400 | 0.0016 | - | | 5.5651 | 6450 | 0.0014 | - | | 5.6083 | 6500 | 0.0015 | - | | 5.6514 | 6550 | 0.0013 | - | | 5.6946 | 6600 | 0.0016 | - | | 5.7377 | 6650 | 0.0016 | - | | 5.7808 | 6700 | 0.0013 | - | | 5.8240 | 6750 | 0.0016 | - | | 5.8671 | 6800 | 0.0019 | - | | 5.9103 | 6850 | 0.0017 | - | | 5.9534 | 6900 | 0.0013 | - | | 5.9965 | 6950 | 0.0019 | - | | 6.0397 | 7000 | 0.0011 | - | | 6.0828 | 7050 | 0.0015 | - | | 6.1260 | 7100 | 0.0015 | - | | 6.1691 | 7150 | 0.0018 | - | | 6.2123 | 7200 | 0.0014 | - | | 6.2554 | 7250 | 0.0014 | - | | 6.2985 | 7300 | 0.0017 | - | | 6.3417 | 7350 | 0.0015 | - | | 6.3848 | 7400 | 0.0017 | - | | 6.4280 | 7450 | 0.0017 | - | | 6.4711 | 7500 | 0.0019 | - | | 6.5142 | 7550 | 0.0017 | - | | 6.5574 | 7600 | 0.0012 | - | | 6.6005 | 7650 | 0.0018 | - | | 6.6437 | 7700 | 0.0015 | - | | 6.6868 | 7750 | 0.002 | - | | 6.7299 | 7800 | 0.0012 | - | | 6.7731 | 7850 | 0.0018 | - | | 6.8162 | 7900 | 0.0014 | - | | 6.8594 | 7950 | 0.0013 | - | | 6.9025 | 8000 | 0.0015 | - | | 6.9456 | 8050 | 0.0015 | - | | 6.9888 | 8100 | 0.0017 | - | | 7.0319 | 8150 | 0.0013 | - | | 7.0751 | 8200 | 0.0017 | - | | 7.1182 | 8250 | 0.0012 | - | | 7.1613 | 8300 | 0.0019 | - | | 7.2045 | 8350 | 0.0013 | - | | 7.2476 | 8400 | 0.0015 | - | | 7.2908 | 8450 | 0.0017 | - | | 7.3339 | 8500 | 0.0016 | - | | 7.3770 | 8550 | 0.0021 | - | | 7.4202 | 8600 | 0.0014 | - | | 7.4633 | 8650 | 0.0013 | - | | 7.5065 | 8700 | 0.0015 | - | | 7.5496 | 8750 | 0.0015 | - | | 7.5928 | 8800 | 0.0014 | - | | 7.6359 | 8850 | 0.0013 | - | | 7.6790 | 8900 | 0.0016 | - | | 7.7222 | 8950 | 0.0016 | - | | 7.7653 | 9000 | 0.0016 | - | | 7.8085 | 9050 | 0.0017 | - | | 7.8516 | 9100 | 0.0016 | - | | 7.8947 | 9150 | 0.0018 | - | | 7.9379 | 9200 | 0.002 | - | | 7.9810 | 9250 | 0.0015 | - | | 8.0242 | 9300 | 0.0015 | - | | 8.0673 | 9350 | 0.0014 | - | | 8.1104 | 9400 | 0.0013 | - | | 8.1536 | 9450 | 0.0014 | - | | 8.1967 | 9500 | 0.0017 | - | | 8.2399 | 9550 | 0.002 | - | | 8.2830 | 9600 | 0.0019 | - | | 8.3261 | 9650 | 0.0012 | - | | 8.3693 | 9700 | 0.0012 | - | | 8.4124 | 9750 | 0.0016 | - | | 8.4556 | 9800 | 0.0014 | - | | 8.4987 | 9850 | 0.0016 | - | | 8.5418 | 9900 | 0.0014 | - | | 8.5850 | 9950 | 0.0012 | - | | 8.6281 | 10000 | 0.0013 | - | | 8.6713 | 10050 | 0.0023 | - | | 8.7144 | 10100 | 0.0011 | - | | 8.7575 | 10150 | 0.0016 | - | | 8.8007 | 10200 | 0.0017 | - | | 8.8438 | 10250 | 0.0017 | - | | 8.8870 | 10300 | 0.0018 | - | | 8.9301 | 10350 | 0.0019 | - | | 8.9733 | 10400 | 0.0017 | - | | 9.0164 | 10450 | 0.0014 | - | | 9.0595 | 10500 | 0.0014 | - | | 9.1027 | 10550 | 0.0012 | - | | 9.1458 | 10600 | 0.0018 | - | | 9.1890 | 10650 | 0.002 | - | | 9.2321 | 10700 | 0.0015 | - | | 9.2752 | 10750 | 0.0019 | - | | 9.3184 | 10800 | 0.0018 | - | | 9.3615 | 10850 | 0.0014 | - | | 9.4047 | 10900 | 0.0016 | - | | 9.4478 | 10950 | 0.0014 | - | | 9.4909 | 11000 | 0.0011 | - | | 9.5341 | 11050 | 0.0014 | - | | 9.5772 | 11100 | 0.0017 | - | | 9.6204 | 11150 | 0.0018 | - | | 9.6635 | 11200 | 0.0012 | - | | 9.7066 | 11250 | 0.0013 | - | | 9.7498 | 11300 | 0.0015 | - | | 9.7929 | 11350 | 0.0019 | - | | 9.8361 | 11400 | 0.0015 | - | | 9.8792 | 11450 | 0.0016 | - | | 9.9223 | 11500 | 0.0013 | - | | 9.9655 | 11550 | 0.0019 | - | ### Framework Versions - Python: 3.10.12 - SetFit: 1.1.0 - Sentence Transformers: 3.2.1 - Transformers: 4.44.2 - PyTorch: 2.5.0+cu121 - Datasets: 3.0.2 - Tokenizers: 0.19.1 ## Citation ### BibTeX ```bibtex @article{https://doi.org/10.48550/arxiv.2209.11055, doi = {10.48550/ARXIV.2209.11055}, url = {https://arxiv.org/abs/2209.11055}, author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren}, keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {Efficient Few-Shot Learning Without Prompts}, publisher = {arXiv}, year = {2022}, copyright = {Creative Commons Attribution 4.0 International} } ```