text_shortening_model_v3

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.4219
• Rouge1: 0.593
• Rouge2: 0.3643
• Rougel: 0.5423
• Rougelsum: 0.5412
• Bert precision: 0.8882
• Bert recall: 0.9022
• Average word count: 11.9
• Max word count: 17
• Min word count: 6
• Average token count: 17.2857

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Average word count	Max word count	Min word count	Average token count
1.6953	1.0	8	1.8235	0.5468	0.3281	0.4997	0.4987	0.8808	0.886	11.5786	18	6	16.8286
1.4749	2.0	16	1.6832	0.5482	0.3138	0.4936	0.4934	0.8776	0.8889	12.1429	18	5	17.2929
1.3967	3.0	24	1.6181	0.5653	0.3362	0.5121	0.512	0.8833	0.894	11.9143	18	5	17.0286
1.3533	4.0	32	1.5757	0.5631	0.338	0.5133	0.5133	0.8838	0.8948	11.8786	18	4	16.9929
1.3	5.0	40	1.5398	0.5748	0.3463	0.5256	0.525	0.8863	0.8977	11.95	18	4	16.9857
1.2528	6.0	48	1.5159	0.58	0.3475	0.5261	0.5247	0.8855	0.8988	11.9571	18	5	17.0429
1.2234	7.0	56	1.4974	0.5823	0.3515	0.5301	0.5289	0.8864	0.8993	11.8929	18	6	17.05
1.2024	8.0	64	1.4819	0.5846	0.3575	0.5326	0.5312	0.8876	0.9014	11.9143	18	6	17.1429
1.1665	9.0	72	1.4680	0.5881	0.3593	0.5367	0.5359	0.8877	0.9014	11.8571	17	6	17.1429
1.1589	10.0	80	1.4567	0.5873	0.359	0.5314	0.5305	0.8873	0.9004	11.7929	17	6	17.0429
1.1411	11.0	88	1.4501	0.5891	0.3627	0.5386	0.5373	0.8888	0.9017	11.85	17	6	17.1286
1.1188	12.0	96	1.4460	0.5911	0.364	0.5399	0.5391	0.8881	0.9024	11.95	17	6	17.2786
1.1061	13.0	104	1.4396	0.5908	0.3648	0.5395	0.5386	0.8881	0.9024	11.9071	17	6	17.3071
1.0939	14.0	112	1.4328	0.5904	0.3625	0.5392	0.5384	0.8876	0.9018	11.9071	17	6	17.3
1.0863	15.0	120	1.4305	0.5899	0.3633	0.5387	0.5379	0.8875	0.9015	11.8714	17	6	17.2714
1.0792	16.0	128	1.4286	0.5908	0.3636	0.5401	0.5392	0.8875	0.9018	11.8929	17	6	17.3
1.0871	17.0	136	1.4255	0.5908	0.3628	0.5401	0.5392	0.8878	0.9017	11.8714	17	6	17.2571
1.057	18.0	144	1.4229	0.5928	0.365	0.5427	0.5414	0.8886	0.9022	11.85	17	6	17.2357
1.0554	19.0	152	1.4221	0.593	0.3643	0.5423	0.5412	0.8882	0.9022	11.9	17	6	17.2857
1.06	20.0	160	1.4219	0.593	0.3643	0.5423	0.5412	0.8882	0.9022	11.9	17	6	17.2857

Framework versions

• Transformers 4.32.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.4
• Tokenizers 0.13.3