how to embed other language documents
#31
by
mahyagh
- opened
Hello everyone! 👋
I’m currently working on a project that involves using Retrieval-Augmented Generation (RAG), and I need to embed documents in various languages (other than English). While there are plenty of great models for English, I'm struggling to find effective solutions for other languages, particularly for languages like Persian, Arabic, or other less commonly supported languages.
If anyone has experience with this or knows of specific models that work well for embedding in different languages, I’d love to hear your recommendations! 🙏
I’m also interested in hearing about any experiences with fine-tuning existing models for non-English languages.