Possibility of new version with all K quants? V2?
Greetings! 👋 🤗
New GGUF quants could be improved if we take what's being discussed here:
https://www.reddit.com/r/LocalLLaMA/comments/1993iro/comment/kie6w9l/
Possibility of new GGUF K quants version for this nice model?
Honestly it's my favorite 7B for roleplay, I prefer it over Kunoichi-7B/Maids even.
Maybe! I'm going to let the quant discussion settle down a bit, this still seems pretty investigatory.
@SanjiWatsuki - Alrighty. Thanks for your work so far! I’m also looking forward to any new developments from you. Stay awesome!
@SanjiWatsuki - Heya! Hope everything is as well as can be.
Using Kalo's general pseudo random text to create the Imatrix data from the F16 model, added additional quants:
https://huggingface.co/Lewdiculous/Loyal-Toppy-Bruins-Maid-7B-DARE-GGUF-Imatrix