gpt2-finetuned-justification-v3

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2415
• Rouge1: 30.8957
• Rouge2: 13.5597
• Rougel: 22.4384
• Rougelsum: 28.2668

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
No log	1.0	338	0.1980	30.0775	13.8145	22.3863	28.0341
0.226	2.0	676	0.1972	28.9676	13.7684	21.8084	26.6768
0.1594	3.0	1014	0.2007	29.8576	13.3727	22.1581	27.5726
0.1594	4.0	1352	0.2071	32.2090	13.7848	22.8787	29.0171
0.1259	5.0	1690	0.2146	28.5240	13.5821	21.4908	26.2550
0.1046	6.0	2028	0.2211	26.1623	13.1641	21.5936	25.0346
0.1046	7.0	2366	0.2294	28.7169	13.4858	21.1068	26.1213
0.0894	8.0	2704	0.2355	30.8957	13.5597	22.4384	28.2668
0.0785	9.0	3042	0.2398	30.8957	13.5597	22.4384	28.2668
0.0785	10.0	3380	0.2415	30.8957	13.5597	22.4384	28.2668

Framework versions

• Transformers 4.36.2
• Pytorch 2.2.2+cu121
• Datasets 2.16.0
• Tokenizers 0.15.2