byt5-base-dv
Pretrained from scratch on Dhivei (language of the Maldives) with ByT5, Google's new byte-level tokenizer strategy.
Use byt5-dv for now; this is less accurate
Corpus: Sofwath's Dhivehi corpus https://github.com/Sofwath/DhivehiDatasets
Pretraining Notebook: https://colab.research.google.com/drive/1ERIZ1PyHn-yN_jo7dTQeODn22vrt-d1d?usp=sharing
Fine-tuning Demo
On Dhivehi news classification task
https://colab.research.google.com/drive/11u5SafR4bKICmArgDl6KQ9vqfYtDpyWp?usp=sharing
Issues
There was an issue with the vocabulary size, final layer, and/or accuracy on fine-tuning.
- Downloads last month
- 18
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.