Hugging Face's logo

yasmineee
/
NLLB_DoRA

PEFT
Safetensors
Generated from Trainer
Model card Files Community
NLLB_DoRA / README.md
yasmineee's picture
yasmineee
finetune-NLLB-600M-on-opus100-Ar2En-with-Dora
14ed422 verified
preview code
|
raw
history blame contribute delete
No virus
1.72 kB
metadata
base_model: facebook/nllb-200-distilled-600M
library_name: peft
license: cc-by-nc-4.0
metrics:
  - bleu
  - rouge
tags:
  - generated_from_trainer
model-index:
  - name: NLLB_DoRA
    results: []

NLLB_DoRA

This model is a fine-tuned version of facebook/nllb-200-distilled-600M on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.2708
  • Bleu: 32.802
  • Rouge: 0.6028
  • Gen Len: 17.4444

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 1
  • eval_batch_size: 1
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 3

Training results

Training Loss Epoch Step Validation Loss Bleu Rouge Gen Len
1.3937 1.0 2000 1.3115 32.2196 0.5954 17.6569
1.3309 2.0 4000 1.2781 32.6752 0.6011 17.4931
1.3234 3.0 6000 1.2708 32.802 0.6028 17.4444

Framework versions

  • PEFT 0.12.0
  • Transformers 4.44.0
  • Pytorch 2.4.0
  • Datasets 2.21.0
  • Tokenizers 0.19.1