wav2vec2-large-xlsr-moroccan-darija-v2

This model is a fine-tuned version of boumehdi/wav2vec2-large-xlsr-moroccan-darija on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2106
• Wer: 0.1908

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
4.8709	0.5359	500	0.4977	0.4531
0.4841	1.0718	1000	0.2715	0.2724
0.3611	1.6077	1500	0.2311	0.2441
0.3088	2.1436	2000	0.2042	0.2366
0.2666	2.6795	2500	0.1999	0.2352
0.2303	3.2154	3000	0.1900	0.2231
0.1921	3.7513	3500	0.1839	0.2195
0.1629	4.2872	4000	0.1783	0.2153
0.1403	4.8232	4500	0.1904	0.2041
0.1178	5.3591	5000	0.1739	0.2118
0.1124	5.8950	5500	0.1996	0.1970
0.0981	6.4309	6000	0.1890	0.2016
0.091	6.9668	6500	0.2020	0.1949
0.077	7.5027	7000	0.2057	0.1929
0.0769	8.0386	7500	0.2093	0.1935
0.0726	8.5745	8000	0.2097	0.1924
0.0685	9.1104	8500	0.2088	0.1913
0.0625	9.6463	9000	0.2106	0.1908

Framework versions

• Transformers 4.43.0.dev0
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1