Anyway to quantize this further down to 4090 level (24Gb VRAM), at Q2_K.gguf level already not sure if it is possible
#1
by
askyforever
- opened
Thanks!
askyforever
changed discussion status to
closed
It's possible but you're not going to have a good time