Correkt/question_extractor_77m

question_extractor

This model is a fine-tuned version of google/flan-t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0926

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
0.7133	0.0973	500	0.2155
0.1448	0.1945	1000	0.1211
0.1325	0.2918	1500	0.1143
0.1338	0.3890	2000	0.1098
0.1256	0.4863	2500	0.1073
0.1259	0.5835	3000	0.1051
0.1238	0.6808	3500	0.1034
0.1188	0.7781	4000	0.1025
0.1157	0.8753	4500	0.1007
0.1187	0.9726	5000	0.0998
0.1135	1.0698	5500	0.0990
0.1114	1.1671	6000	0.0985
0.1141	1.2643	6500	0.0973
0.1106	1.3616	7000	0.0969
0.1119	1.4589	7500	0.0962
0.1126	1.5561	8000	0.0961
0.1076	1.6534	8500	0.0955
0.1113	1.7506	9000	0.0951
0.1097	1.8479	9500	0.0947
0.1098	1.9451	10000	0.0943
0.1082	2.0424	10500	0.0941
0.1079	2.1397	11000	0.0939
0.1056	2.2369	11500	0.0938
0.1064	2.3342	12000	0.0936
0.1053	2.4314	12500	0.0933
0.1085	2.5287	13000	0.0931
0.1062	2.6259	13500	0.0931
0.1094	2.7232	14000	0.0929
0.1081	2.8205	14500	0.0930
0.1051	2.9177	15000	0.0930

Framework versions

• Transformers 4.40.2
• Pytorch 2.2.1+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1