t5-finetuned-summarization-samsum

This model is a fine-tuned version of google-t5/t5-small on the samsum dataset. It achieves the following results on the evaluation set:

• Loss: 1.6551
• Rouge1: 43.6894
• Rouge2: 21.0711
• Rougel: 36.7865
• Rougelsum: 40.2927

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
2.0612	1.0	1842	1.7709	40.7189	17.9391	34.0848	37.86
1.8988	2.0	3684	1.7278	41.1985	18.7817	34.8297	38.378
1.8283	3.0	5526	1.6946	42.5298	19.6906	35.7159	39.2425
1.7798	4.0	7368	1.6860	42.9966	20.7335	36.5141	39.7994
1.7418	5.0	9210	1.6677	42.8533	20.4738	36.1407	39.5548
1.7157	6.0	11052	1.6645	43.6738	21.055	36.8091	40.3053
1.6896	7.0	12894	1.6584	43.5629	20.8972	36.614	40.2316
1.6756	8.0	14736	1.6567	43.8709	21.4421	36.9208	40.5036
1.6624	9.0	16578	1.6568	43.6278	21.0048	36.668	40.2666
1.6558	10.0	18420	1.6551	43.6894	21.0711	36.7865	40.2927

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1