flan-t5-large-invoice

This model is a fine-tuned version of google/flan-t5-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.7345
• Rouge1: 20.6872
• Rouge2: 8.404
• Rougel: 16.4285
• Rougelsum: 18.8189
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.779	1.0	933	1.3862	19.5863	7.7096	15.7946	18.3055	19.0
1.3799	2.0	1866	1.3274	19.6876	6.7441	15.804	17.8499	19.0
1.2505	3.0	2799	1.3099	19.8912	7.0276	16.3826	17.8505	19.0
1.1726	4.0	3732	1.3136	20.9896	8.5214	16.7944	19.2926	19.0
1.0985	5.0	4665	1.3205	20.5499	7.9224	16.3301	18.519	19.0
1.0439	6.0	5598	1.3297	20.8897	7.6685	16.6002	19.1149	19.0
0.9912	7.0	6531	1.3515	21.0205	7.9648	16.6663	19.1507	19.0
0.922	8.0	7464	1.3819	20.0508	7.0989	16.2521	18.2977	19.0
0.8729	9.0	8397	1.4222	20.3315	8.1219	16.5236	18.8624	19.0
0.8353	10.0	9330	1.4372	19.3277	7.4633	15.5985	17.6354	19.0
0.7928	11.0	10263	1.4851	20.1063	7.9988	16.0191	18.3571	19.0
0.7561	12.0	11196	1.5084	20.6322	7.2781	16.3066	18.8342	19.0
0.7251	13.0	12129	1.5437	21.4814	8.1047	17.0497	19.8034	19.0
0.692	14.0	13062	1.5802	21.1577	8.0934	16.738	19.2418	19.0
0.668	15.0	13995	1.6349	19.8489	7.7261	15.8122	18.0971	19.0
0.6237	16.0	14928	1.6246	20.8915	8.2484	16.8627	19.0533	19.0
0.6166	17.0	15861	1.6662	20.3532	8.2927	16.2535	18.5981	19.0
0.5973	18.0	16794	1.6963	20.4374	7.9296	16.2445	18.4927	19.0
0.581	19.0	17727	1.7168	20.6517	8.3175	16.4803	18.8024	19.0
0.5704	20.0	18660	1.7345	20.6872	8.404	16.4285	18.8189	19.0

Framework versions

• Transformers 4.43.2
• Pytorch 2.2.0a0+81ea7a4
• Datasets 2.20.0
• Tokenizers 0.19.1