Can we have a GPTQ 4bit quantized version for consumer grade hardware?
#1
by
teefan
- opened
Thanks for the model! What's a great work. I would like to try, but it's too large, can we have a quantized model of it, like in 4bit with group sizes? That would be awesome to have!
Okay sure... I will try to quantize the model to 4bit , and share it to everyone soon... Thanks for suggestion