nyankole_wav2vec2

This model is a fine-tuned version of facebook/wav2vec2-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.8606
• Wer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 55
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
7.4272	0.4739	50	2.9131	1.0
2.8729	0.9479	100	2.8567	1.0
2.9027	1.4218	150	2.9014	1.0
2.8676	1.8957	200	2.8607	1.0
2.8484	2.3697	250	2.8482	1.0
2.8807	2.8436	300	2.8553	1.0
2.8513	3.3175	350	2.8731	1.0
2.8489	3.7915	400	2.8524	1.0
2.841	4.2654	450	2.8468	1.0
2.9182	4.7393	500	2.8572	1.0
2.8441	5.2133	550	2.8642	1.0
2.8486	5.6872	600	2.8690	1.0
2.8455	6.1611	650	2.8663	1.0
2.845	6.6351	700	2.8611	1.0
2.856	7.1090	750	2.8781	1.0
2.8429	7.5829	800	2.8612	1.0
2.8456	8.0569	850	2.8555	1.0
2.8465	8.5308	900	2.8596	1.0
2.8465	9.0047	950	2.8507	1.0
2.848	9.4787	1000	2.8527	1.0
2.8441	9.9526	1050	2.8702	1.0
2.8452	10.4265	1100	2.8782	1.0
2.8446	10.9005	1150	2.8736	1.0
2.8433	11.3744	1200	2.8541	1.0
2.842	11.8483	1250	2.8678	1.0
2.8442	12.3223	1300	2.8559	1.0
2.8473	12.7962	1350	2.8538	1.0
2.843	13.2701	1400	2.8592	1.0
2.8429	13.7441	1450	2.8571	1.0
2.8431	14.2180	1500	2.8860	1.0
2.8428	14.6919	1550	2.8611	1.0
2.8429	15.1659	1600	2.8763	1.0
2.8387	15.6398	1650	2.8637	1.0
2.8503	16.1137	1700	2.8588	1.0
2.8463	16.5877	1750	2.8560	1.0
2.8414	17.0616	1800	2.8550	1.0
2.8417	17.5355	1850	2.8582	1.0
2.8438	18.0095	1900	2.8502	1.0
2.8497	18.4834	1950	2.8825	1.0
2.8377	18.9573	2000	2.8622	1.0
2.8412	19.4313	2050	2.8711	1.0
2.8405	19.9052	2100	2.8786	1.0
2.8419	20.3791	2150	2.8467	1.0
2.8426	20.8531	2200	2.8627	1.0
2.8454	21.3270	2250	2.8640	1.0
2.8397	21.8009	2300	2.8600	1.0
2.8405	22.2749	2350	2.8716	1.0
2.8413	22.7488	2400	2.8498	1.0
2.8454	23.2227	2450	2.8647	1.0
2.8415	23.6967	2500	2.8727	1.0
2.8381	24.1706	2550	2.8600	1.0
2.8405	24.6445	2600	2.8604	1.0
2.8442	25.1185	2650	2.8543	1.0
2.836	25.5924	2700	2.8613	1.0
2.8479	26.0664	2750	2.8664	1.0
2.842	26.5403	2800	2.8574	1.0
2.8406	27.0142	2850	2.8558	1.0
2.8435	27.4882	2900	2.8587	1.0
2.8387	27.9621	2950	2.8568	1.0
2.8442	28.4360	3000	2.8573	1.0
2.8365	28.9100	3050	2.8591	1.0
2.8428	29.3839	3100	2.8621	1.0
2.8386	29.8578	3150	2.8606	1.0

Framework versions

• Transformers 4.41.2
• Pytorch 2.1.2
• Datasets 2.19.2
• Tokenizers 0.19.1