GGUF Issues
#1
by
leonardlin
- opened
When testing the GGUF, I found that it would start outputting garbo a few turns in. I've filed a bug here, but I believe it's related to some previous Llama 3 issues that llama.cpp has had, either in conversion or in inferencing... https://github.com/ggerganov/llama.cpp/issues/7513
Please only download these GGUF files if you're working on debugging these issues.
Apparently the issues I had were due to the way that the server handles context:
leonardlin
changed discussion status to
closed