metadata

license: apache-2.0
base_model: google/electra-small-discriminator
tags:
  - generated_from_keras_callback
model-index:
  - name: nguyennghia0902/electra-small-discriminator_0.0001_32_15e
    results: []

nguyennghia0902/electra-small-discriminator_0.0001_32_15e

This model is a fine-tuned version of google/electra-small-discriminator on an unknown dataset. It achieves the following results on the evaluation set:

Train Loss: 0.7520
Train End Logits Accuracy: 0.7904
Train Start Logits Accuracy: 0.7699
Validation Loss: 0.3342
Validation End Logits Accuracy: 0.9048
Validation Start Logits Accuracy: 0.9021
Epoch: 11

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 0.0001, 'decay_steps': 23445, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
training_precision: float32

Training results

Train Loss	Train End Logits Accuracy	Train Start Logits Accuracy	Validation Loss	Validation End Logits Accuracy	Validation Start Logits Accuracy	Epoch
3.0253	0.3302	0.2968	2.2414	0.4704	0.4533	0
2.3162	0.4597	0.4260	1.8267	0.5511	0.5364	1
2.0285	0.5160	0.4813	1.5472	0.6109	0.5994	2
1.8125	0.5587	0.5287	1.2995	0.6688	0.6512	3
1.6192	0.5963	0.5677	1.0973	0.7105	0.7030	4
1.4482	0.6341	0.6066	0.8998	0.7637	0.7547	5
1.2931	0.6694	0.6423	0.7622	0.7920	0.7916	6
1.1518	0.6980	0.6741	0.6412	0.8260	0.8197	7
1.0351	0.7240	0.7025	0.5316	0.8518	0.8531	8
0.9269	0.7488	0.7270	0.4671	0.8701	0.8700	9
0.8354	0.7714	0.7489	0.3836	0.8910	0.8896	10
0.7520	0.7904	0.7699	0.3342	0.9048	0.9021	11

Framework versions

Transformers 4.39.3
TensorFlow 2.15.0
Datasets 2.18.0
Tokenizers 0.15.2