Transformers
GGUF
English
mistral

The best model

#1
by Kenshiro-28 - opened

This is the best model I have tested, thank you very much :)

One thing, from time to time I get this tag in the response: </|im_end|> . It happened also in the previous version of Dolphin. It's a malformed end tag, maybe it's in the training data.

In the previous version only happened in GPT4ALL UI.
LM Studio seem to handle it better.

I tried many quantisations and many settings.

On LM Studio Choose ChatML prompt preset image.png

In the 2.1 version using ChatML I would get things like </|im_end|> and then it would start talking as if it was me etc? It never happened if I used the other things only ChatML. If I choose Alpaca or Guanaco it was fine. What mode should I be using it in? I use it in instruct mode? I like having long sets of instructions and asking multiple questions. Should I use chat instead?

I update my oobabooga but could my ChatML be broken out outdated? Do I need to add a custom one? I'm not sure if this problem is fixed or not for 2.2.

I'm using it with MAGI, the model works fine, the only issue is that from time to time you get a </|im_end|> in the response, but functionality is ok.

https://github.com/Kenshiro-28/MAGI

Sign up or log in to comment