mlx-community

/

dbrx-instruct-4bit

like 48

The chat template should be ChatML.

When I run it as-is I get this:

When I manually add ChatML I get proper output.

When I use Dolphin 8x7b without manually adding ChatML, it's added (probably because of the tokenizer config)

I am guessing (not being an expert in mlx) what you gotta do, is add it to tokenizer_config.json

"chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",