tgrhn/wav2vec2-turkish-4

Wav2Vec2-XLS-TR

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the Common Voice 17 dataset. It achieves the following results on the evaluation set:

• Loss: 0.4817

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
8.5104	1.0	1451	5.2696
3.7233	2.0	2902	3.2482
2.6473	3.0	4353	1.6343
1.0373	4.0	5804	0.8273
0.6737	5.0	7255	0.6109
0.5356	6.0	8706	0.5244
0.4556	7.0	10157	0.4773
0.4099	8.0	11608	0.4491
0.3648	9.0	13059	0.4378
0.3426	10.0	14510	0.4343
0.3226	11.0	15961	0.4279
0.3033	12.0	17412	0.4336
0.2861	13.0	18863	0.4195
0.2683	14.0	20314	0.4282
0.2514	15.0	21765	0.4204
0.2439	16.0	23216	0.4388
0.2368	17.0	24667	0.4325
0.2263	18.0	26118	0.4402
0.2162	19.0	27569	0.4470
0.2092	20.0	29020	0.4427
0.207	21.0	30471	0.4590
0.2014	22.0	31922	0.4605
0.1954	23.0	33373	0.4682
0.1895	24.0	34824	0.4683
0.1858	25.0	36275	0.4657
0.1792	26.0	37726	0.4692
0.1782	27.0	39177	0.4728
0.1793	28.0	40628	0.4783
0.1743	29.0	42079	0.4782
0.1732	30.0	43530	0.4817

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1