t5-abs-1609-1450-lr-0.0001-bs-10-maxep-20

This model is a fine-tuned version of google-t5/t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.0493
• Rouge/rouge1: 0.4186
• Rouge/rouge2: 0.188
• Rouge/rougel: 0.3713
• Rouge/rougelsum: 0.3708
• Bertscore/bertscore-precision: 0.9077
• Bertscore/bertscore-recall: 0.8772
• Bertscore/bertscore-f1: 0.8921
• Meteor: 0.338
• Gen Len: 31.5

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 10
• eval_batch_size: 10
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 20
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge/rouge1	Rouge/rouge2	Rouge/rougel	Rouge/rougelsum	Bertscore/bertscore-precision	Bertscore/bertscore-recall	Bertscore/bertscore-f1	Meteor	Gen Len
4.4242	0.8	2	2.9745	0.24	0.0922	0.1926	0.1916	0.8591	0.8535	0.8562	0.1759	42.0
2.7301	2.0	5	2.5893	0.2451	0.074	0.2114	0.2133	0.8517	0.8543	0.8526	0.1844	42.4
3.7041	2.8	7	2.3562	0.3045	0.114	0.2614	0.2633	0.8785	0.8587	0.8683	0.2344	36.2
2.2169	4.0	10	2.2206	0.3076	0.1129	0.2568	0.2569	0.9027	0.8623	0.8819	0.2244	25.4
3.0954	4.8	12	2.1692	0.3291	0.1429	0.2855	0.2857	0.9089	0.8684	0.8881	0.2405	24.1
1.8983	6.0	15	2.1128	0.3455	0.1237	0.2864	0.2872	0.9022	0.8674	0.8843	0.2511	25.2
2.6884	6.8	17	2.0867	0.3451	0.1133	0.2873	0.2881	0.9021	0.8683	0.8847	0.2509	27.1
1.7137	8.0	20	2.0663	0.3424	0.1218	0.2932	0.2945	0.8963	0.8711	0.8833	0.2819	31.1
2.4552	8.8	22	2.0603	0.3491	0.1272	0.2932	0.2948	0.8975	0.8714	0.884	0.2834	30.6
1.5859	10.0	25	2.0565	0.3502	0.1207	0.2962	0.2979	0.8952	0.8675	0.8809	0.2635	29.8
2.2768	10.8	27	2.0558	0.3606	0.1253	0.3021	0.3031	0.8951	0.8683	0.8813	0.2725	30.4
1.4516	12.0	30	2.0541	0.4032	0.1573	0.3355	0.3358	0.9024	0.8744	0.8881	0.3035	32.3
2.1365	12.8	32	2.0514	0.4087	0.1714	0.3445	0.3448	0.9038	0.8749	0.889	0.32	32.8
1.4049	14.0	35	2.0507	0.4167	0.1792	0.3515	0.3524	0.9065	0.8765	0.8911	0.3222	31.9
2.0878	14.8	37	2.0498	0.4167	0.1792	0.3515	0.3524	0.9065	0.8765	0.8911	0.3222	31.9
1.361	16.0	40	2.0493	0.4186	0.188	0.3713	0.3708	0.9077	0.8772	0.8921	0.338	31.5

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1