he-cantillation
This model is a fine-tuned version of openai/whisper-tiny on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.1635
- Wer: 15.8869
- Avg Precision Exact: 0.8555
- Avg Recall Exact: 0.8550
- Avg F1 Exact: 0.8548
- Avg Precision Letter Shift: 0.8805
- Avg Recall Letter Shift: 0.8801
- Avg F1 Letter Shift: 0.8799
- Avg Precision Word Level: 0.8850
- Avg Recall Word Level: 0.8845
- Avg F1 Word Level: 0.8842
- Avg Precision Word Shift: 0.9658
- Avg Recall Word Shift: 0.9665
- Avg F1 Word Shift: 0.9656
- Precision Median Exact: 0.9333
- Recall Median Exact: 0.9333
- F1 Median Exact: 0.9375
- Precision Max Exact: 1.0
- Recall Max Exact: 1.0
- F1 Max Exact: 1.0
- Precision Min Exact: 0.0
- Recall Min Exact: 0.0
- F1 Min Exact: 0.0
- Precision Min Letter Shift: 0.0
- Recall Min Letter Shift: 0.0
- F1 Min Letter Shift: 0.0
- Precision Min Word Level: 0.0
- Recall Min Word Level: 0.0
- F1 Min Word Level: 0.0
- Precision Min Word Shift: 0.1429
- Recall Min Word Shift: 0.1111
- F1 Min Word Shift: 0.125
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 50
- training_steps: 20000
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | Wer | Avg Precision Exact | Avg Recall Exact | Avg F1 Exact | Avg Precision Letter Shift | Avg Recall Letter Shift | Avg F1 Letter Shift | Avg Precision Word Level | Avg Recall Word Level | Avg F1 Word Level | Avg Precision Word Shift | Avg Recall Word Shift | Avg F1 Word Shift | Precision Median Exact | Recall Median Exact | F1 Median Exact | Precision Max Exact | Recall Max Exact | F1 Max Exact | Precision Min Exact | Recall Min Exact | F1 Min Exact | Precision Min Letter Shift | Recall Min Letter Shift | F1 Min Letter Shift | Precision Min Word Level | Recall Min Word Level | F1 Min Word Level | Precision Min Word Shift | Recall Min Word Shift | F1 Min Word Shift |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
No log | 8e-05 | 1 | 8.6830 | 101.5891 | 0.0009 | 0.0016 | 0.0010 | 0.0100 | 0.0077 | 0.0077 | 0.0054 | 0.0173 | 0.0076 | 0.0465 | 0.0380 | 0.0371 | 0.0 | 0.0 | 0.0 | 0.125 | 0.3333 | 0.1667 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0958 | 0.32 | 4000 | 0.1924 | 25.9534 | 0.7565 | 0.7583 | 0.7566 | 0.7917 | 0.7936 | 0.7919 | 0.7983 | 0.8005 | 0.7986 | 0.9281 | 0.9324 | 0.9293 | 0.8571 | 0.8571 | 0.8571 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0539 | 0.64 | 8000 | 0.1813 | 21.8625 | 0.7983 | 0.7994 | 0.7983 | 0.8279 | 0.8291 | 0.8279 | 0.8342 | 0.8359 | 0.8344 | 0.9437 | 0.9468 | 0.9445 | 0.9091 | 0.9091 | 0.9032 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1111 | 0.125 |
0.0353 | 0.96 | 12000 | 0.1755 | 19.0909 | 0.8271 | 0.8284 | 0.8273 | 0.8558 | 0.8572 | 0.8560 | 0.8610 | 0.8624 | 0.8611 | 0.9537 | 0.9561 | 0.9542 | 0.9167 | 0.9167 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1111 | 0.125 |
0.0197 | 1.28 | 16000 | 0.1664 | 17.0288 | 0.8452 | 0.8444 | 0.8444 | 0.8711 | 0.8705 | 0.8703 | 0.8762 | 0.8756 | 0.8754 | 0.9609 | 0.9625 | 0.9610 | 0.9286 | 0.9286 | 0.9286 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1111 | 0.125 |
0.0088 | 1.6 | 20000 | 0.1635 | 15.8869 | 0.8555 | 0.8550 | 0.8548 | 0.8805 | 0.8801 | 0.8799 | 0.8850 | 0.8845 | 0.8842 | 0.9658 | 0.9665 | 0.9656 | 0.9333 | 0.9333 | 0.9375 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1111 | 0.125 |
Framework versions
- Transformers 4.41.2
- Pytorch 2.2.1
- Datasets 2.20.0
- Tokenizers 0.19.1
- Downloads last month
- 4
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for cantillation/Teamim-tiny_Random-True_DropOut-0.1_WeightDecay-1e-06_Augmented_date-27-06-2024_09-48-25
Base model
openai/whisper-tiny
Finetuned
this model