SetFit with ppsingh/SECTOR-multilabel-mpnet_w
This is a SetFit model that can be used for Text Classification. This SetFit model uses ppsingh/SECTOR-multilabel-mpnet_w as the Sentence Transformer embedding model. A SetFitHead instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
- Model Type: SetFit
- Sentence Transformer body: ppsingh/SECTOR-multilabel-mpnet_w
- Classification head: a SetFitHead instance
- Maximum Sequence Length: 512 tokens
- Number of Classes: 4 classes
Model Sources
- Repository: SetFit on GitHub
- Paper: Efficient Few-Shot Learning Without Prompts
- Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("ppsingh/iki_sector_setfit")
# Run inference
preds = model("In the shipping and aviation sectors, emission reduction efforts will be focused on distributing eco-friendly ships and enhancing the operational efficiency of aircraft. Agriculture, livestock farming and fisheries: The Republic Korea is introducing various options to accelerate low-carbon farming, for instance, improving irrigation techniques in rice paddies and adopting low-input systems for nitrogen fertilizers.")
Training Details
Training Set Metrics
Training set | Min | Median | Max |
---|---|---|---|
Word count | 35 | 76.164 | 170 |
Training Dataset: 250
Class Positive Count of Class Economy-wide 88 Energy 63 Other Sector 64 Transport 139 Validation Dataset: 42
Class Positive Count of Class Economy-wide 15 Energy 11 Other Sector 11 Transport 24
Training Hyperparameters
- batch_size: (16, 2)
- num_epochs: (1, 10)
- max_steps: -1
- sampling_strategy: oversampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.01
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False
Training Results
Epoch | Step | Training Loss | Validation Loss |
---|---|---|---|
0.0005 | 1 | 0.2029 | - |
0.0993 | 200 | 0.0111 | 0.1124 |
0.1985 | 400 | 0.0063 | 0.111 |
0.2978 | 600 | 0.0183 | 0.1214 |
0.3970 | 800 | 0.0197 | 0.1248 |
0.4963 | 1000 | 0.0387 | 0.1339 |
0.5955 | 1200 | 0.0026 | 0.1181 |
0.6948 | 1400 | 0.0378 | 0.1208 |
0.7940 | 1600 | 0.0285 | 0.1267 |
0.8933 | 1800 | 0.0129 | 0.1254 |
0.9926 | 2000 | 0.0341 | 0.1271 |
Classifier Training Results
Epoch | Training F1-micro | Training F1-Samples | Training F1-weighted | Validation F1-micro | Validation F1-samples | Validation F1-weighted |
---|---|---|---|---|---|---|
0 | 0.954 | 0.972 | 0.945 | 0.824 | 0.819 | 0.813 |
1 | 0.994 | 0.996 | 0.994 | 0.850 | 0.832 | 0.852 |
2 | 0.981 | 0.989 | 0.979 | 0.850 | 0.843 | 0.852 |
3 | 0.995 | 0.997 | 0.995 | 0.852 | 0.843 | 0.858 |
4 | 0.994 | 0.996 | 0.994 | 0.852 | 0.843 | 0.858 |
5 | 0.995 | 0.997 | 0.995 | 0.859 | 0.848 | 0.863 |
label | precision | recall | f1-score | support |
---|---|---|---|---|
Economy-wide | 0.857 | 0.800 | 0.827 | 15.0 |
Energy | 1.00 | 0.818 | 0.900 | 11.0 |
Other Sector | 0.615 | 0.727 | 0.667 | 11.0 |
Transport | 0.958 | 0.958 | 0.958 | 24.0 |
- Micro Avg: Precision = 0.866, Recall = 0.852, F1 = 0.859504
- Samples Avg: Precision = 0.869, Recall = 0.861, F1 = 0.848
Environmental Impact
Carbon emissions were measured using CodeCarbon.
- Carbon Emitted: 0.026 kg of CO2
- Hours Used: 0.622 hours
Training Hardware
- On Cloud: No
- GPU Model: 1 x Tesla T4
- CPU Model: Intel(R) Xeon(R) CPU @ 2.00GHz
- RAM Size: 12.67 GB
Framework Versions
- Python: 3.10.12
- SetFit: 1.0.3
- Sentence Transformers: 2.3.1
- Transformers: 4.35.2
- PyTorch: 2.1.0+cu121
- Datasets: 2.3.0
- Tokenizers: 0.15.1
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
- Downloads last month
- 12
Inference API (serverless) has been turned off for this model.
Model tree for ppsingh/iki_sector_setfit
Base model
GIZ/SECTOR-multilabel-mpnet_w