https://huggingface.co/NLPark/AnFeng_v3_Avocet
A fine-tune of Command-R-01
Need imat quants.
fail to create a static quant:
'''Error: Error converting to fp16: b'Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\nTraceback (most recent call last):\n File "/home/user/app/llama.cpp/convert-hf-to-gguf.py", line 3001, in \n main()\n File "/home/user/app/llama.cpp/convert-hf-to-gguf.py", line 2988, in main\n model_instance.set_vocab()\n File "/home/user/app/llama.cpp/convert-hf-to-gguf.py", line 75, in set_vocab\n self._set_vocab_gpt2()\n File "/home/user/app/llama.cpp/convert-hf-to-gguf.py", line 331, in _set_vocab_gpt2\n tokens, toktypes, tokpre = self.get_vocab_base()\n File "/home/user/app/llama.cpp/convert-hf-to-gguf.py", line 242, in get_vocab_base\n tokpre = self.get_vocab_base_pre(tokenizer)\n File "/home/user/app/llama.cpp/convert-hf-to-gguf.py", line 323, in get_vocab_base_pre\n raise NotImplementedError("BPE pre-tokenizer was not recognized - update get_vocab_base_pre()")\nNotImplementedError: BPE pre-tokenizer was not recognized - update get_vocab_base_pre()\n'
'''
The llama-3 hacks that were recently added broke command-r support (thats what you see). I'll try to convetr it using an older version.
The llama-3 hacks that were recently added broke command-r support (thats what you see). I'll try to convetr it using an older version.
haha, it's so funny.
1 bug solved caused ∞ bugs growth.
quantizing has become a nightmare in recent weeks. can't even laugh about it anymore. anyway, the staic q2uants are finished, and the imatrrix ones willo start being generated soon, if nothing goes wrong :)
oh, and some engines will now warn about reduced quality, because the pretokenizer is not set by convert.py.
the imatrix quants are now also being generated and should be done in a few hours. have fun!