tgrhn/wav2vec2-turkish-3

Wav2Vec2-XLS-TR

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the Common Voice 17 dataset. It achieves the following results on the evaluation set:

• Loss: 2.7124

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
23.6246	0.3446	500	16.2983
8.2656	0.6892	1000	5.9544
5.1083	1.0338	1500	4.6067
4.1663	1.3784	2000	3.8176
3.559	1.7229	2500	3.4058
3.2582	2.0675	3000	3.2021
2.9475	2.4121	3500	3.0052
1.9117	2.7567	4000	2.7124
1.2089	3.1013	4500	2.6678
0.9505	3.4459	5000	3.1079
0.7688	3.7905	5500	2.6448
0.6474	4.1351	6000	2.1442
0.5717	4.4797	6500	2.6866
0.5262	4.8243	7000	2.7227
0.4764	5.1688	7500	2.5182
0.4452	5.5134	8000	2.7089
0.4237	5.8580	8500	2.4970
0.4004	6.2026	9000	2.0918
0.3845	6.5472	9500	2.1695
0.3685	6.8918	10000	2.3281
0.3582	7.2364	10500	2.0681
0.337	7.5810	11000	2.2537
0.3301	7.9256	11500	2.0853
0.3142	8.2702	12000	2.2379
0.3087	8.6147	12500	2.0322
0.3026	8.9593	13000	2.3212
0.2894	9.3039	13500	2.3748
0.2828	9.6485	14000	2.0785
0.2798	9.9931	14500	1.9353
0.262	10.3377	15000	2.2790
0.2592	10.6823	15500	2.4833
0.2613	11.0269	16000	2.4773
0.2484	11.3715	16500	2.0992
0.246	11.7161	17000	2.3585
0.2423	12.0606	17500	2.3101
0.2341	12.4052	18000	2.2959
0.231	12.7498	18500	1.9466
0.2283	13.0944	19000	2.0529
0.224	13.4390	19500	2.2572
0.226	13.7836	20000	2.2684
0.2157	14.1282	20500	2.0767
0.2128	14.4728	21000	2.4035
0.2117	14.8174	21500	2.1512
0.2062	15.1620	22000	2.2322
0.2002	15.5065	22500	2.1550
0.2033	15.8511	23000	2.5139
0.1997	16.1957	23500	2.1999
0.1942	16.5403	24000	2.5342
0.1942	16.8849	24500	2.5350
0.1923	17.2295	25000	2.6199
0.1869	17.5741	25500	2.5196
0.1875	17.9187	26000	2.3316
0.1825	18.2633	26500	2.4622
0.1828	18.6079	27000	2.3920
0.1808	18.9524	27500	2.4571
0.1797	19.2970	28000	2.5495
0.1777	19.6416	28500	2.1355
0.1752	19.9862	29000	2.4399
0.1723	20.3308	29500	2.4222
0.1719	20.6754	30000	2.7283
0.1702	21.0200	30500	2.4584
0.1655	21.3646	31000	2.4646
0.1658	21.7092	31500	2.3659
0.1676	22.0538	32000	2.5923
0.1606	22.3983	32500	2.6588
0.1572	22.7429	33000	2.3206
0.1611	23.0875	33500	2.4420
0.156	23.4321	34000	2.6875
0.1587	23.7767	34500	2.6309
0.1596	24.1213	35000	2.4329
0.1572	24.4659	35500	2.6451
0.1552	24.8105	36000	2.6626
0.1529	25.1551	36500	2.6210
0.155	25.4997	37000	2.6027
0.1531	25.8442	37500	2.7159
0.1564	26.1888	38000	2.6926
0.1497	26.5334	38500	2.8263
0.1443	26.8780	39000	2.6340
0.15	27.2226	39500	2.6501
0.1456	27.5672	40000	2.7202
0.1489	27.9118	40500	2.6929
0.15	28.2564	41000	2.7965
0.147	28.6010	41500	2.6591
0.1471	28.9456	42000	2.7030
0.1473	29.2901	42500	2.6906
0.1458	29.6347	43000	2.7225
0.1438	29.9793	43500	2.7124

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1