Edit model card

slurp-slot_baseline-xlm_r-en

This model is a fine-tuned version of xlm-roberta-base on the SLURP dataset.

It achieves the following results on the test set:

  • Loss: 0.3263
  • Precision: 0.7954
  • Recall: 0.8413
  • F1: 0.8177
  • Accuracy: 0.9268

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Precision Recall F1 Accuracy
1.1437 1.0 720 0.5236 0.6852 0.6623 0.6736 0.8860
0.5761 2.0 1440 0.3668 0.7348 0.7829 0.7581 0.9119
0.3087 3.0 2160 0.2996 0.7925 0.8280 0.8099 0.9270
0.2631 4.0 2880 0.2959 0.7872 0.8487 0.8168 0.9275
0.1847 5.0 3600 0.3121 0.7929 0.8373 0.8145 0.9290
0.1518 6.0 4320 0.3117 0.8080 0.8601 0.8332 0.9329
0.1232 7.0 5040 0.3153 0.7961 0.8490 0.8217 0.9267
0.0994 8.0 5760 0.3125 0.8105 0.8570 0.8331 0.9332
0.0968 9.0 6480 0.3242 0.8147 0.8637 0.8385 0.9329
0.0772 10.0 7200 0.3263 0.8145 0.8641 0.8386 0.9341

Test results per slot

slot f1 tc_size
alarm_type 0.4 4
app_name 0.42857142857142855 10
artist_name 0.8122605363984675 123
audiobook_author 0.0 9
audiobook_name 0.6021505376344087 43
business_name 0.8530259365994236 184
business_type 0.6666666666666667 41
change_amount 0.6666666666666666 9
coffee_type 0.5333333333333333 6
color_type 0.8135593220338982 28
cooking_type 0.8333333333333333 14
currency_name 0.8611111111111112 70
date 0.9034267912772587 623
definition_word 0.88 97
device_type 0.8053691275167785 71
drink_type 0.0 2
email_address 0.9599999999999999 38
email_folder 0.9523809523809523 10
event_name 0.7643504531722054 321
food_type 0.7482014388489208 121
game_name 0.7789473684210527 44
general_frequency 0.5862068965517242 21
house_place 0.8840579710144928 68
ingredient 0.0 13
joke_type 0.9411764705882353 17
list_name 0.7979274611398963 91
meal_type 0.782608695652174 18
media_type 0.8596491228070176 173
movie_name 0.0 3
movie_type 0.5 3
music_album 0.0 2
music_descriptor 0.25 8
music_genre 0.7244094488188977 58
news_topic 0.5675675675675675 64
order_type 0.7941176470588235 29
person 0.9128094725511302 438
personal_info 0.6666666666666666 16
place_name 0.8725790010193679 493
player_setting 0.5405405405405405 42
playlist_name 0.5 27
podcast_descriptor 0.4888888888888888 28
podcast_name 0.5245901639344263 31
radio_name 0.6504065040650406 53
relation 0.8478260869565218 87
song_name 0.7058823529411765 54
time 0.7914893617021276 236
time_zone 0.7804878048780488 23
timeofday 0.8396946564885496 60
transport_agency 0.8571428571428571 18
transport_descriptor 0.0 2
transport_name 0.4 7
transport_type 0.9481481481481482 68
weather_descriptor 0.789272030651341 123

Framework versions

  • Transformers 4.28.1
  • Pytorch 2.0.0+cu118
  • Datasets 2.11.0
  • Tokenizers 0.13.3
Downloads last month
82
Safetensors
Model size
278M params
Tensor type
I64
·
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.