How did you quantised this model?
#3
by
Saksham54
- opened
So, i also have fine tuned the gemma-7b on my personal dataset, but now i wanna convert it to a .gguf file to run on ollama for some usecase, can you tell me how did you quantised it, because the llama.cpp doesnt support gemma for quantization
I've quantized this model with llama.cpp in fact. They also officially support it as I see in their model list.