Svetlana0303's picture
Upload TFAlbertForSequenceClassification
7c27f91
metadata
license: apache-2.0
tags:
  - generated_from_keras_callback
model-index:
  - name: Regression_albert_aug_CustomLoss_3
    results: []

Regression_albert_aug_CustomLoss_3

This model is a fine-tuned version of albert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

  • Train Loss: 0.2368
  • Train Mae: 0.5301
  • Train Mse: 0.4296
  • Train R2-score: 0.7669
  • Validation Loss: 0.2410
  • Validation Mae: 0.5680
  • Validation Mse: 0.4286
  • Validation R2-score: 0.6930
  • Epoch: 14

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 1e-04, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
  • training_precision: float32

Training results

Train Loss Train Mae Train Mse Train R2-score Validation Loss Validation Mae Validation Mse Validation R2-score Epoch
0.2614 0.5480 0.4524 0.7369 0.2408 0.5194 0.4609 0.7578 0
0.2442 0.5374 0.4362 0.7109 0.2334 0.5376 0.4391 0.7399 1
0.2431 0.5349 0.4356 0.7503 0.2432 0.5234 0.4657 0.7591 2
0.2386 0.5250 0.4264 0.7926 0.2348 0.5525 0.4316 0.7203 3
0.2409 0.5342 0.4325 0.7166 0.2431 0.5233 0.4656 0.7591 4
0.2400 0.5298 0.4310 0.7553 0.2358 0.5250 0.4490 0.7513 5
0.2384 0.5274 0.4299 0.7791 0.2341 0.5491 0.4329 0.7253 6
0.2413 0.5306 0.4335 0.7593 0.2365 0.5583 0.4299 0.7109 7
0.2381 0.5299 0.4298 0.7784 0.2335 0.5452 0.4347 0.7306 8
0.2379 0.5280 0.4297 0.7575 0.2335 0.5448 0.4349 0.7312 9
0.2374 0.5306 0.4309 0.8098 0.2334 0.5441 0.4352 0.7321 10
0.2381 0.5302 0.4303 0.7428 0.2337 0.5466 0.4340 0.7288 11
0.2376 0.5323 0.4275 0.7806 0.2333 0.5411 0.4369 0.7358 12
0.2339 0.5277 0.4217 0.7986 0.2363 0.5232 0.4506 0.7525 13
0.2368 0.5301 0.4296 0.7669 0.2410 0.5680 0.4286 0.6930 14

Framework versions

  • Transformers 4.28.1
  • TensorFlow 2.12.0
  • Datasets 2.12.0
  • Tokenizers 0.13.3