BaoPhanDuy/Model_QA_NLP

This model is a fine-tuned version of distilbert/distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 4.0943
• Validation Loss: 4.1522
• Epoch: 19

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 500, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
4.7034	4.2730	0
4.1720	4.1522	1
4.0999	4.1522	2
4.0938	4.1522	3
4.0993	4.1522	4
4.1005	4.1522	5
4.0951	4.1522	6
4.0896	4.1522	7
4.0938	4.1522	8
4.0921	4.1522	9
4.1018	4.1522	10
4.0922	4.1522	11
4.1055	4.1522	12
4.1004	4.1522	13
4.0996	4.1522	14
4.0920	4.1522	15
4.1055	4.1522	16
4.0896	4.1522	17
4.0894	4.1522	18
4.0943	4.1522	19

Framework versions

• Transformers 4.42.4
• TensorFlow 2.17.0
• Datasets 2.20.0
• Tokenizers 0.19.1