SECTOR-multilabel-climatebert
This model is a fine-tuned version of climatebert/distilroberta-base-climate-f on the Policy-Classification dataset.
The loss function BCEWithLogitsLoss is modified with pos_weight to focus on recall, therefore instead of loss the evaluation metrics are used to assess the model performance during training It achieves the following results on the evaluation set:
- Loss: 0.6028
- Precision-micro: 0.6395
- Precision-samples: 0.7543
- Precision-weighted: 0.6475
- Recall-micro: 0.7762
- Recall-samples: 0.8583
- Recall-weighted: 0.7762
- F1-micro: 0.7012
- F1-samples: 0.7655
- F1-weighted: 0.7041
Model description
The purpose of this model is to predict multiple labels simultaneously from a given input data. Specifically, the model will predict Sector labels - Agriculture,Buildings, Coastal Zone,Cross-Cutting Area,Disaster Risk Management (DRM),Economy-wide,Education,Energy,Environment,Health,Industries,LULUCF/Forestry,Social Development,Tourism, Transport,Urban,Waste,Water
Intended uses & limitations
More information needed
Training and evaluation data
Training Dataset: 10123
Class Positive Count of Class Agriculture 2235 Buildings 169 Coastal Zone 698 Cross-Cutting Area 1853 Disaster Risk Management (DRM) 814 Economy-wide 873 Education 180 Energy 2847 Environment 905 Health 662 Industries 419 LULUCF/Forestry 1861 Social Development 507 Tourism 192 Transport 1173 Urban 558 Waste 714 Water 1207 Validation Dataset: 936
Class Positive Count of Class Agriculture 200 Buildings 18 Coastal Zone 71 Cross-Cutting Area 180 Disaster Risk Management (DRM) 85 Economy-wide 85 Education 23 Energy 254 Environment 91 Health 68 Industries 41 LULUCF/Forestry 193 Social Development 56 Tourism 28 Transport 107 Urban 51 Waste 59 Water 106
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 9.07e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 300
- num_epochs: 7
Training results
Training Loss | Epoch | Step | Validation Loss | Precision-micro | Precision-samples | Precision-weighted | Recall-micro | Recall-samples | Recall-weighted | F1-micro | F1-samples | F1-weighted |
---|---|---|---|---|---|---|---|---|---|---|---|---|
0.6978 | 1.0 | 633 | 0.5968 | 0.3948 | 0.5274 | 0.4982 | 0.7873 | 0.8675 | 0.7873 | 0.5259 | 0.5996 | 0.5793 |
0.485 | 2.0 | 1266 | 0.5255 | 0.5089 | 0.6365 | 0.5469 | 0.7984 | 0.8749 | 0.7984 | 0.6216 | 0.6907 | 0.6384 |
0.3657 | 3.0 | 1899 | 0.5248 | 0.4984 | 0.6617 | 0.5397 | 0.8141 | 0.8769 | 0.8141 | 0.6183 | 0.7066 | 0.6393 |
0.2585 | 4.0 | 2532 | 0.5457 | 0.5807 | 0.7148 | 0.5992 | 0.8007 | 0.8752 | 0.8007 | 0.6732 | 0.7449 | 0.6813 |
0.1841 | 5.0 | 3165 | 0.5551 | 0.6016 | 0.7426 | 0.6192 | 0.7937 | 0.8677 | 0.7937 | 0.6844 | 0.7590 | 0.6917 |
0.1359 | 6.0 | 3798 | 0.5913 | 0.6349 | 0.7506 | 0.6449 | 0.7844 | 0.8676 | 0.7844 | 0.7018 | 0.7667 | 0.7057 |
0.1133 | 7.0 | 4431 | 0.6028 | 0.6395 | 0.7543 | 0.6475 | 0.7762 | 0.8583 | 0.7762 | 0.7012 | 0.7655 | 0.7041 |
label | precision | recall | f1-score | support |
---|---|---|---|---|
Agriculture | 0.720 | 0.850 | 0.780 | 200 |
Buildings | 0.636 | 0.777 | 0.700 | 18 |
Coastal Zone | 0.562 | 0.760 | 0.646 | 71 |
Cross-Cutting Area | 0.569 | 0.777 | 0.657 | 180 |
Disaster Risk Management (DRM) | 0.567 | 0.694 | 0.624 | 85 |
Economy-wide | 0.461 | 0.635 | 0.534 | 85 |
Education | 0.608 | 0.608 | 0.608 | 23 |
Energy | 0.816 | 0.838 | 0.827 | 254 |
Environment | 0.561 | 0.703 | 0.624 | 91 |
Health | 0.708 | 0.750 | 0.728 | 68 |
Industries | 0.660 | 0.902 | 0.762 | 41 |
LULUCF/Forestry | 0.676 | 0.844 | 0.751 | 193 |
Social Development | 0.593 | 0.678 | 0.633 | 56 |
Tourism | 0.551 | 0.571 | 0.561 | 28 |
Transport | 0.700 | 0.766 | 0.732 | 107 |
Urban | 0.414 | 0.568 | 0.479 | 51 |
Waste | 0.658 | 0.881 | 0.753 | 59 |
Water | 0.602 | 0.773 | 0.677 | 106 |
Environmental Impact
Carbon emissions were measured using CodeCarbon.
- Carbon Emitted: 0.02867 kg of CO2
- Hours Used: 0.706 hours
Training Hardware
- On Cloud: yes
- GPU Model: 1 x Tesla T4
- CPU Model: Intel(R) Xeon(R) CPU @ 2.00GHz
- RAM Size: 12.67 GB
Framework versions
- Transformers 4.38.1
- Pytorch 2.1.0+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2
- Downloads last month
- 9
Model tree for GIZ/SECTOR-multilabel-climatebert_f
Base model
climatebert/distilroberta-base-climate-f