Llama3-ChatQA-1.5-8B / tokenizer.json

Commit History

The tokenizer adds a special token '<|im_end|>' to solve the problem of non-stop generation when encountering <|im_end|>.
a31949e
verified

zjyhf commited on

Upload 8 files
d3fe95e
verified

zihanliu commited on