Hugging Face's logo

roequitz
/
t5-abs-2309-1054-lr-1e-05-bs-10-maxep-20

Safetensors
t5
Generated from Trainer
Model card Files Community
t5-abs-2309-1054-lr-1e-05-bs-10-maxep-20 / README.md
roequitz's picture
roequitz
End of training
2102305 verified
preview code
|
raw
history blame contribute delete
No virus
6.41 kB
metadata
license: apache-2.0
base_model: google-t5/t5-base
tags:
  - generated_from_trainer
model-index:
  - name: t5-abs-2309-1054-lr-1e-05-bs-10-maxep-20
    results: []

t5-abs-2309-1054-lr-1e-05-bs-10-maxep-20

This model is a fine-tuned version of google-t5/t5-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 4.0044
  • Rouge/rouge1: 0.4791
  • Rouge/rouge2: 0.2351
  • Rouge/rougel: 0.4085
  • Rouge/rougelsum: 0.4098
  • Bertscore/bertscore-precision: 0.8984
  • Bertscore/bertscore-recall: 0.8999
  • Bertscore/bertscore-f1: 0.899
  • Meteor: 0.447
  • Gen Len: 41.7727

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 10
  • eval_batch_size: 10
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 20
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 20
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Rouge/rouge1 Rouge/rouge2 Rouge/rougel Rouge/rougelsum Bertscore/bertscore-precision Bertscore/bertscore-recall Bertscore/bertscore-f1 Meteor Gen Len
0.0089 0.9885 43 3.9871 0.4806 0.2393 0.4125 0.4129 0.8987 0.8999 0.8991 0.4493 41.7
0.0062 2.0 87 3.9921 0.4805 0.2395 0.4119 0.4124 0.8984 0.8997 0.8989 0.4488 41.6727
0.0046 2.9885 130 3.9973 0.4806 0.2358 0.4101 0.4109 0.8984 0.8993 0.8988 0.448 41.2
0.0046 4.0 174 4.0023 0.4787 0.2353 0.4084 0.4095 0.8989 0.899 0.8988 0.4445 40.8273
0.0051 4.9885 217 4.0062 0.4817 0.2381 0.4116 0.4125 0.8996 0.8992 0.8993 0.4456 40.5455
0.0044 6.0 261 4.0107 0.4796 0.2351 0.4089 0.4099 0.8994 0.8988 0.8989 0.4423 40.2727
0.0046 6.9885 304 4.0121 0.4795 0.2331 0.4083 0.409 0.8991 0.8986 0.8987 0.4393 40.1455
0.0043 8.0 348 4.0119 0.4799 0.2345 0.4086 0.4092 0.899 0.899 0.8989 0.4426 40.6909
0.0043 8.9885 391 4.0124 0.4778 0.2344 0.4076 0.4083 0.899 0.8988 0.8988 0.4402 40.5364
0.0038 10.0 435 4.0146 0.4791 0.2349 0.4087 0.4096 0.8992 0.8991 0.899 0.4413 40.6909
0.0113 10.9885 478 4.0149 0.4794 0.2361 0.4088 0.4096 0.8985 0.8992 0.8987 0.4436 41.2091
0.0116 12.0 522 4.0099 0.4817 0.2387 0.4112 0.4119 0.8986 0.8997 0.899 0.4482 41.5545
0.0116 12.9885 565 4.0083 0.4811 0.2378 0.411 0.4119 0.8992 0.8997 0.8993 0.4472 41.3636
0.0109 14.0 609 4.0073 0.4804 0.2363 0.41 0.4108 0.899 0.8998 0.8993 0.4462 41.4364
0.0109 14.9885 652 4.0056 0.4796 0.2362 0.409 0.4096 0.8987 0.9 0.8992 0.4476 41.7636
0.0107 16.0 696 4.0045 0.4796 0.2353 0.4095 0.4098 0.8988 0.8998 0.8991 0.4471 41.4727
0.0117 16.9885 739 4.0039 0.4789 0.234 0.4076 0.4084 0.8992 0.8997 0.8993 0.4455 41.2455
0.0103 18.0 783 4.0045 0.4785 0.2342 0.4078 0.4088 0.8986 0.8996 0.899 0.4452 41.4909
0.0111 18.9885 826 4.0044 0.4776 0.2339 0.4069 0.408 0.8986 0.8998 0.8991 0.4456 41.6
0.0119 19.7701 860 4.0044 0.4791 0.2351 0.4085 0.4098 0.8984 0.8999 0.899 0.447 41.7727

Framework versions

  • Transformers 4.44.0
  • Pytorch 2.4.0
  • Datasets 2.21.0
  • Tokenizers 0.19.1