GIZ
/

Edit model card

SECTOR-multilabel-climatebert

This model is a fine-tuned version of climatebert/distilroberta-base-climate-f on the Policy-Classification dataset.

The loss function BCEWithLogitsLoss is modified with pos_weight to focus on recall, therefore instead of loss the evaluation metrics are used to assess the model performance during training It achieves the following results on the evaluation set:

  • Loss: 0.6028
  • Precision-micro: 0.6395
  • Precision-samples: 0.7543
  • Precision-weighted: 0.6475
  • Recall-micro: 0.7762
  • Recall-samples: 0.8583
  • Recall-weighted: 0.7762
  • F1-micro: 0.7012
  • F1-samples: 0.7655
  • F1-weighted: 0.7041

Model description

The purpose of this model is to predict multiple labels simultaneously from a given input data. Specifically, the model will predict Sector labels - Agriculture,Buildings, Coastal Zone,Cross-Cutting Area,Disaster Risk Management (DRM),Economy-wide,Education,Energy,Environment,Health,Industries,LULUCF/Forestry,Social Development,Tourism, Transport,Urban,Waste,Water

Intended uses & limitations

More information needed

Training and evaluation data

  • Training Dataset: 10123

    Class Positive Count of Class
    Agriculture 2235
    Buildings 169
    Coastal Zone 698
    Cross-Cutting Area 1853
    Disaster Risk Management (DRM) 814
    Economy-wide 873
    Education 180
    Energy 2847
    Environment 905
    Health 662
    Industries 419
    LULUCF/Forestry 1861
    Social Development 507
    Tourism 192
    Transport 1173
    Urban 558
    Waste 714
    Water 1207
  • Validation Dataset: 936

    Class Positive Count of Class
    Agriculture 200
    Buildings 18
    Coastal Zone 71
    Cross-Cutting Area 180
    Disaster Risk Management (DRM) 85
    Economy-wide 85
    Education 23
    Energy 254
    Environment 91
    Health 68
    Industries 41
    LULUCF/Forestry 193
    Social Development 56
    Tourism 28
    Transport 107
    Urban 51
    Waste 59
    Water 106

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 9.07e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 300
  • num_epochs: 7

Training results

Training Loss Epoch Step Validation Loss Precision-micro Precision-samples Precision-weighted Recall-micro Recall-samples Recall-weighted F1-micro F1-samples F1-weighted
0.6978 1.0 633 0.5968 0.3948 0.5274 0.4982 0.7873 0.8675 0.7873 0.5259 0.5996 0.5793
0.485 2.0 1266 0.5255 0.5089 0.6365 0.5469 0.7984 0.8749 0.7984 0.6216 0.6907 0.6384
0.3657 3.0 1899 0.5248 0.4984 0.6617 0.5397 0.8141 0.8769 0.8141 0.6183 0.7066 0.6393
0.2585 4.0 2532 0.5457 0.5807 0.7148 0.5992 0.8007 0.8752 0.8007 0.6732 0.7449 0.6813
0.1841 5.0 3165 0.5551 0.6016 0.7426 0.6192 0.7937 0.8677 0.7937 0.6844 0.7590 0.6917
0.1359 6.0 3798 0.5913 0.6349 0.7506 0.6449 0.7844 0.8676 0.7844 0.7018 0.7667 0.7057
0.1133 7.0 4431 0.6028 0.6395 0.7543 0.6475 0.7762 0.8583 0.7762 0.7012 0.7655 0.7041
label precision recall f1-score support
Agriculture 0.720 0.850 0.780 200
Buildings 0.636 0.777 0.700 18
Coastal Zone 0.562 0.760 0.646 71
Cross-Cutting Area 0.569 0.777 0.657 180
Disaster Risk Management (DRM) 0.567 0.694 0.624 85
Economy-wide 0.461 0.635 0.534 85
Education 0.608 0.608 0.608 23
Energy 0.816 0.838 0.827 254
Environment 0.561 0.703 0.624 91
Health 0.708 0.750 0.728 68
Industries 0.660 0.902 0.762 41
LULUCF/Forestry 0.676 0.844 0.751 193
Social Development 0.593 0.678 0.633 56
Tourism 0.551 0.571 0.561 28
Transport 0.700 0.766 0.732 107
Urban 0.414 0.568 0.479 51
Waste 0.658 0.881 0.753 59
Water 0.602 0.773 0.677 106

Environmental Impact

Carbon emissions were measured using CodeCarbon.

  • Carbon Emitted: 0.02867 kg of CO2
  • Hours Used: 0.706 hours

Training Hardware

  • On Cloud: yes
  • GPU Model: 1 x Tesla T4
  • CPU Model: Intel(R) Xeon(R) CPU @ 2.00GHz
  • RAM Size: 12.67 GB

Framework versions

  • Transformers 4.38.1
  • Pytorch 2.1.0+cu121
  • Datasets 2.18.0
  • Tokenizers 0.15.2
Downloads last month
11
Safetensors
Model size
82.3M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for GIZ/SECTOR-multilabel-climatebert_f

Finetuned
(6)
this model

Dataset used to train GIZ/SECTOR-multilabel-climatebert_f