---
tags:
- summarization
- generated_from_trainer
metrics:
- rouge
model-index:
- name: exp2-led-risalah_data_v6
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/silmiaulia/huggingface/runs/7pt54hkh)
# exp2-led-risalah_data_v6

This model was trained from scratch on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.7971
- Rouge1: 35.2105
- Rouge2: 14.2825
- Rougel: 18.7356
- Rougelsum: 33.8518

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 4
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 200
- num_epochs: 30
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|
| 3.6579        | 1.0   | 20   | 2.9715          | 15.0902 | 2.4602  | 8.8016  | 14.54     |
| 3.8322        | 2.0   | 40   | 2.6400          | 19.8395 | 3.3421  | 10.0041 | 18.8629   |
| 3.3928        | 3.0   | 60   | 2.4780          | 24.1438 | 4.9177  | 11.8553 | 23.0439   |
| 3.1159        | 4.0   | 80   | 2.3336          | 26.1339 | 5.4015  | 12.1066 | 24.5407   |
| 2.8469        | 5.0   | 100  | 2.2554          | 25.3388 | 5.6665  | 12.0706 | 24.1799   |
| 2.6486        | 6.0   | 120  | 2.1842          | 33.8164 | 9.2363  | 15.7673 | 31.606    |
| 2.5429        | 7.0   | 140  | 2.1322          | 32.5361 | 8.5141  | 15.3201 | 30.935    |
| 2.3159        | 8.0   | 160  | 2.0631          | 32.3657 | 9.171   | 15.1179 | 30.9634   |
| 2.1821        | 10.0  | 200  | 1.9358          | 33.1626 | 10.8072 | 16.5887 | 31.0652   |
| 2.2141        | 11.0  | 220  | 1.9274          | 36.3525 | 13.5885 | 18.4941 | 34.9263   |
| 2.1213        | 12.0  | 240  | 1.9033          | 34.4359 | 11.4335 | 17.8322 | 32.5781   |
| 1.9791        | 13.0  | 260  | 1.8914          | 37.0733 | 14.2739 | 18.9338 | 35.5985   |
| 1.9504        | 14.0  | 280  | 1.8642          | 34.7529 | 13.0325 | 18.1055 | 33.257    |
| 1.9848        | 15.0  | 300  | 1.8641          | 35.9266 | 13.4528 | 18.459  | 34.0294   |
| 1.845         | 16.0  | 320  | 1.8507          | 37.7424 | 15.2488 | 18.993  | 35.4955   |
| 1.8049        | 17.0  | 340  | 1.8390          | 36.5023 | 13.6069 | 18.4956 | 34.883    |
| 1.8158        | 18.0  | 360  | 1.8393          | 34.4722 | 13.6438 | 18.1636 | 32.4511   |
| 1.8541        | 19.0  | 380  | 1.8395          | 37.0215 | 14.3221 | 19.6743 | 35.3083   |
| 1.7967        | 20.0  | 400  | 1.8403          | 36.3048 | 13.3475 | 19.9887 | 34.6884   |
| 1.7285        | 21.0  | 420  | 1.8394          | 36.4051 | 14.3198 | 19.4997 | 34.9803   |
| 1.7303        | 22.0  | 440  | 1.8287          | 36.1003 | 14.166  | 17.8619 | 34.3505   |
| 1.6976        | 23.0  | 460  | 1.8040          | 34.3036 | 12.8173 | 18.6643 | 32.6019   |
| 1.6916        | 24.0  | 480  | 1.7963          | 34.7753 | 14.0332 | 18.923  | 33.3743   |
| 1.6872        | 25.0  | 500  | 1.8073          | 37.0718 | 14.6821 | 20.1188 | 35.7824   |
| 1.6979        | 26.0  | 520  | 1.8340          | 37.1726 | 15.1384 | 20.2153 | 36.3188   |
| 1.6867        | 27.0  | 540  | 1.8000          | 37.2831 | 14.2806 | 19.1448 | 36.1598   |
| 1.6959        | 28.0  | 560  | 1.7886          | 34.8414 | 13.5902 | 18.5803 | 33.5383   |
| 1.7546        | 29.0  | 580  | 1.8068          | 37.6551 | 16.1055 | 20.2492 | 36.1177   |
| 1.632         | 30.0  | 600  | 1.7971          | 35.2105 | 14.2825 | 18.7356 | 33.8518   |


### Framework versions

- Transformers 4.42.3
- Pytorch 2.1.2
- Datasets 2.20.0
- Tokenizers 0.19.1