Equall

/

Saul-7B-Instruct-v1

like 74

Hi,

I was trying to inference using this model and it seems the model works only for the llama2 jinja template and the results seem to be comparatively low. I tried mistral template, the model goes horribly wrong and when I remove the chat template completely, it gives me issues saying,

"jinja2.exceptions.TemplateError: Conversation roles must alternate user/assistant/user/assistant/..."

It seems like I cannot pass a system prompt in this case. my prompts order is; system and then user thats it.

Following is my code :

//some function

Can you suggest anything to get better running settings ?