Good news, they are working!
#2
by
appvoid
- opened
These models now work with the new dolly's ggml implementation found here: https://github.com/ggerganov/ggml/tree/master/examples/dolly-v2
I just needed to make some small modifications since by default it includes dolly v2's training prompt and change end of text token to as in the original pythia models.
This is so cool!
Btw, 70m and 160m doesn't worth it even on q5_1, I suggest using models after 160m parameters.