JuliusFx/dyu-fr-t5-base_v1

This model is a fine-tuned version of t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 1.3233
• Validation Loss: 3.0376
• Epoch: 46

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
3.2963	3.1402	0
3.0454	3.0436	1
2.9282	3.0219	2
2.8338	2.9804	3
2.7463	2.9778	4
2.6807	2.9307	5
2.6156	2.9288	6
2.5501	2.9221	7
2.4933	2.9245	8
2.4400	2.9083	9
2.3910	2.9285	10
2.3451	2.9178	11
2.2967	2.9217	12
2.2496	2.9160	13
2.2099	2.9176	14
2.1726	2.8832	15
2.1317	2.9009	16
2.0931	2.8764	17
2.0541	2.8484	18
2.0237	2.8875	19
1.9935	2.8943	20
1.9538	2.8810	21
1.9218	2.8885	22
1.8905	2.8650	23
1.8631	2.8671	24
1.8290	2.8832	25
1.8046	2.8879	26
1.7761	2.9429	27
1.7414	2.9406	28
1.7167	2.9296	29
1.6926	2.9174	30
1.6639	2.9762	31
1.6421	2.9700	32
1.6102	2.9565	33
1.5877	2.9810	34
1.5658	2.9643	35
1.5390	3.0225	36
1.5152	3.0029	37
1.4990	2.9756	38
1.4748	3.0228	39
1.4483	3.0092	40
1.4286	3.0356	41
1.4051	3.0226	42
1.3841	3.0442	43
1.3577	3.0595	44
1.3433	3.0547	45
1.3233	3.0376	46

Framework versions

• Transformers 4.38.2
• TensorFlow 2.15.0
• Datasets 2.18.0
• Tokenizers 0.15.2