EOS token is getting printed
what client is this?
It's marked as not being special in your tokenizer_config.json which, from my experience, means that the engine doesn't know it's a special token that it's not meant to display
It's FreeChat, on Mac.
Ah.. Okay, I have no idea why. Is it too late to fix this once it's quantized? Like, is there a way to edit the GGUF file?
You can edit the metadata manually to fix it, I can also just remake them properly
Okay. I have edited the tokenizer_config.json here: https://huggingface.co/migtissera/Tess-v2.5-Phi-3-medium-128k-14B/blob/main/tokenizer_config.json
Can you do a small quant, like maybe an 8-bit, and then we can check it before you go ahead and make all the quants?
@migtissera uploaded fixed q8 to test
Okay, downloading now
Please go ahead with rest of the quantizations. Thanks for your support and responsiveness!