text_shortening_model_v61

This model is a fine-tuned version of t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7370
• Rouge1: 0.6559
• Rouge2: 0.469
• Rougel: 0.6075
• Rougelsum: 0.6079
• Bert precision: 0.9075
• Bert recall: 0.9017
• Bert f1-score: 0.9041
• Average word count: 7.9152
• Max word count: 15
• Min word count: 3
• Average token count: 12.1741
• % shortened texts with length > 12: 6.6964

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Bert f1-score	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
2.2731	1.0	49	1.3305	0.3966	0.2328	0.3397	0.3396	0.7258	0.7385	0.7316	9.3438	19	0	16.3929	28.5714
1.3225	2.0	98	0.9829	0.6051	0.422	0.5558	0.5557	0.8863	0.879	0.8822	8.0491	17	0	12.6607	8.0357
1.0933	3.0	147	0.8678	0.6346	0.4487	0.5869	0.5875	0.9012	0.8928	0.8965	7.8527	15	0	12.1607	5.8036
0.9836	4.0	196	0.8145	0.6404	0.449	0.5911	0.5918	0.9034	0.8971	0.8997	8.0179	15	3	12.1964	8.4821
0.9182	5.0	245	0.7860	0.647	0.4598	0.597	0.5974	0.9055	0.8989	0.9017	7.8884	15	3	12.1116	7.1429
0.8756	6.0	294	0.7659	0.6479	0.4606	0.5999	0.5996	0.9054	0.8982	0.9013	7.8839	15	3	12.1205	7.1429
0.84	7.0	343	0.7517	0.6544	0.4688	0.6062	0.6061	0.9067	0.9008	0.9033	7.9196	15	3	12.1741	7.1429
0.8256	8.0	392	0.7424	0.6515	0.4644	0.6033	0.6033	0.9068	0.9001	0.903	7.8705	15	3	12.1473	6.25
0.8198	9.0	441	0.7386	0.656	0.469	0.6076	0.608	0.9076	0.9017	0.9041	7.9107	15	3	12.1696	6.6964
0.8058	10.0	490	0.7370	0.6559	0.469	0.6075	0.6079	0.9075	0.9017	0.9041	7.9152	15	3	12.1741	6.6964

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3