Failed to create LLM 'zephyr' from '/models/zephyr-7b-alpha.Q5_K_M.gguf'.
#6
by
whoknowsmeinhf
- opened
Added model_type='zephyr' in CTransformers.
File "llm.py", line 6, in <module>
llm = AutoModelForCausalLM.from_pretrained("zephyr-7b-alpha.Q5_K_M.gguf",model_type="zephyr",seed=0)
File "/usr/local/lib/python3.9/site-packages/ctransformers-0.2.23-py3.9-linux-x86_64.egg/ctransformers/hub.py", line 178, in from_pretrained
return LLM(
File "/usr/local/lib/python3.9/site-packages/ctransformers-0.2.23-py3.9-linux-x86_64.egg/ctransformers/llm.py", line 234, in __init__
raise RuntimeError(