Arbi-Houssem/TunLangModel_test1.12

Whisper Tunisien

This model is a fine-tuned version of openai/whisper-small on the Tunisian_dataset_STT-TTS15s_filtred1.0 dataset. It achieves the following results on the evaluation set:

• Loss: 2.9037
• Wer: 104.9291

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-07
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 15000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
1.2369	7.7519	1000	3.2094	117.6907
1.0283	15.5039	2000	3.0674	110.6685
0.9629	23.2558	3000	3.0180	130.3174
0.893	31.0078	4000	2.9887	126.7387
0.835	38.7597	5000	2.9676	103.5111
0.7907	46.5116	6000	2.9500	107.2248
0.7624	54.2636	7000	2.9370	107.5625
0.7624	62.0155	8000	2.9270	104.4564
0.7198	69.7674	9000	2.9200	104.3889
0.6818	77.5194	10000	2.9143	111.6138
0.7245	85.2713	11000	2.9099	104.5240
0.6762	93.0233	12000	2.9071	104.5915
0.6691	100.7752	13000	2.9052	104.8616
0.6366	108.5271	14000	2.9040	104.2539
0.6801	116.2791	15000	2.9037	104.9291

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1