mistralai

/

Mistral-7B-v0.1

like 3.39k

It giving the response that could not really understand the input data.

Hi coder could you share a bit more about your datasets and how you fine tuned the model?

i used the unsloth notebook to train the mistral models ! <<
so you can use the alpaca format : or change it to suit your needs , in this case i owuld train the model on some instruct ( alpaca ) datasets : then when outputting the model to GGUF to make sure you first train the model with a Q/A prompt : As the gguf GPT4all and software uses this :
BUT:
If your going to use it as a openAI type model ( you need to train it with the chat ml format before you output it ) this model was not trained with the ChatML template so if you want to use tools or funciton calling then : train using the chatml format and gguf it like this :

If you want to dump a lot of knowledge into the model then you would use the text completion method : ( then it will generate text well ) ---
NOTE: because you trained it in any of these mode: you just need to choose your method to output the model at the end in gguf :

so After you have trained your model : then you can edit your tokenizer.config and remove an underlaying templates : then train for alignment for a few step to implement the correct template for youor desired usage : ( then it will imprint this on to the model ) :
then when you output the model and host it local or on server it will respond as expected !

Really for software its a Q/A format and for servers its ChatML
SO probably make the same model and output both variations :
this strategy allows you to train for anything ! then align the model to your desired prompt and output template .. the save as gguf :

the best qwhich has worked the cheepest and well on colab(free) was the unsloth ( to run local you need python 3.10( windows 50/50 ) - Linnx 100% ( all need cuda setup working )

Runs fine i the free colab ! >>> Unsloth also keep updating the notebooks they share ! << so as they change the server setups and update the bitsandbytes and peft etc it moves with them so if you have problems just get a new notebook as they are maintaied !>>>

its easy to miss a stage !! Alignment is important !<<<<<< KEY !

It giving the response that could not really understand the input data.

Hi coder could you share a bit more about your datasets and how you fine tuned the model?

My dataset is in the form of
##Instruction:
##Response:
And for Fine tuning model, I uses peft, lora