MaziyarPanahi

/

Llama-3-8B-Instruct-v0.8

like 8

Hi @DeepMount00

No, it's not a merge with any model outside my own fine-tuned models. Back then we didn't have that many 8B models unlike Mistral 7B, so I fine-tuned my own. If I remember correctly, this specific model is a merge between v0.4 and another version of my own model and then it went through a DPO fine-tuning.

I have moved on to making qwen2 models, but I can try to do some merging with VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct and other models. (now that we have more 8B models out there)

Thank you for your response, @MaziyarPanahi . I am curious about the dataset you used for training this model. It performs exceptionally well in the Italian language, and since I am conducting research in this area, I would appreciate it if you could share the dataset you used.

This is very interesting! I've never noticed nor intended to improve multi-lingual capability of any of my fine-tuned models. The answers in Italian language, is it just better in terms of writing style and vibe or is more correct compare to the 8B Instruct?

I used some public DPO datasets, and if I am not mistaken 1 private one that I know it's multi-lingual, but never noticed that it might improve other languages.

I compared your model with Llama 8B Instruct, and found that your model performs better in the Italian language in several key areas:

• The Italian text generation is more grammatically and semantically accurate.
• It has a superior understanding of sentence nuances, resulting in more appropriate and relevant responses.

Could you please provide the name of the public dataset used for fine-tuning?

interesting! never noticed that, thanks for sharing.

Since it's a merge of multiple fine-tuned models of mine, the public ones are the usual ones:

• Orca
• OpenHermes
• Ultrafeedback
• Math