Template ? not able t use it.
what should be the template ? what should i selected in lm studio ? could you provide ollama models too?
@bartowski could you provide better version of gguf model which can support lmstudio/jan etc...
chatml - and theres one on ollama as well.
chatml is not working @Crystalcareai
I'll take a look though these ones were made by me so it's odd it's giving issues
@NK-Spacewalk are you using flash attention? Qwen models trend to prefer it and sometimes will misbehave without it
@bartowski
Thank you for comment.
Im not using flash attention. im using latest lm studio.
this gguf file is not working with LM Studio, jan.
but ollama model is working , which i created manually by using same gguf file.
You should turn flash attention on, that will fix it. It's possible ollama is doing this by default
I've had good success with this spark variant in lm-studio using chatml template: