t5-abs-2309-1054-lr-0.0001-bs-2-maxep-20

This model is a fine-tuned version of google-t5/t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 4.1236
• Rouge/rouge1: 0.4731
• Rouge/rouge2: 0.2208
• Rouge/rougel: 0.3994
• Rouge/rougelsum: 0.4008
• Bertscore/bertscore-precision: 0.8972
• Bertscore/bertscore-recall: 0.897
• Bertscore/bertscore-f1: 0.897
• Meteor: 0.4314
• Gen Len: 40.8273

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge/rouge1	Rouge/rouge2	Rouge/rougel	Rouge/rougelsum	Bertscore/bertscore-precision	Bertscore/bertscore-recall	Bertscore/bertscore-f1	Meteor	Gen Len
0.049	1.0	217	3.0726	0.4642	0.2147	0.395	0.3945	0.8959	0.897	0.8963	0.4246	41.0364
0.025	2.0	434	3.5278	0.4809	0.2331	0.4126	0.4135	0.8997	0.9002	0.8998	0.44	40.2545
0.0177	3.0	651	3.7709	0.4672	0.2131	0.3912	0.3918	0.8974	0.8961	0.8966	0.4224	40.0
0.014	4.0	868	3.8327	0.4738	0.2244	0.4005	0.4009	0.8966	0.8985	0.8974	0.4361	42.1364
0.0278	5.0	1085	3.8865	0.4679	0.2181	0.3942	0.3949	0.8968	0.8983	0.8974	0.4296	41.5909
0.0246	6.0	1302	3.8697	0.4642	0.2147	0.3904	0.3915	0.8959	0.8976	0.8966	0.421	41.6818
0.0204	7.0	1519	3.9737	0.4646	0.2159	0.395	0.3953	0.8964	0.8967	0.8964	0.421	40.7273
0.0179	8.0	1736	4.0367	0.461	0.2102	0.3896	0.3904	0.8969	0.8946	0.8956	0.4122	38.9727
0.0158	9.0	1953	4.0384	0.4695	0.2117	0.391	0.3921	0.8975	0.8978	0.8976	0.4269	40.4455
0.0159	10.0	2170	4.0446	0.4672	0.2166	0.3945	0.3951	0.8966	0.8982	0.8972	0.4296	41.3091
0.0126	11.0	2387	4.0704	0.4722	0.2223	0.3966	0.3979	0.8968	0.8978	0.8972	0.4356	41.1636
0.0132	12.0	2604	4.1046	0.468	0.2207	0.4011	0.402	0.8974	0.8978	0.8975	0.4341	40.5636
0.0109	13.0	2821	4.1023	0.4743	0.2217	0.4	0.4003	0.8978	0.8971	0.8974	0.4311	40.6091
0.0106	14.0	3038	4.1477	0.4691	0.2202	0.3979	0.3984	0.8974	0.8963	0.8967	0.4257	40.3545
0.0103	15.0	3255	4.1412	0.4753	0.2219	0.4048	0.4063	0.8982	0.8967	0.8973	0.4247	39.5091
0.01	16.0	3472	4.1251	0.4762	0.2259	0.4045	0.4063	0.8983	0.8978	0.898	0.4337	40.3909
0.0087	17.0	3689	4.1286	0.482	0.2256	0.405	0.4063	0.8971	0.8985	0.8976	0.4449	41.6455
0.0092	18.0	3906	4.1284	0.4675	0.2185	0.3981	0.3993	0.897	0.8973	0.897	0.4288	41.0818
0.0089	19.0	4123	4.1252	0.4695	0.2182	0.3981	0.3991	0.8966	0.897	0.8967	0.427	41.0636
0.0081	20.0	4340	4.1236	0.4731	0.2208	0.3994	0.4008	0.8972	0.897	0.897	0.4314	40.8273

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1