ValueError: Failed to load model from file: models\gemma-2-27b-it-Q3_K_M.gguf
Just downloaded Q4K_M quant and got this error. Any way to fix this?
Traceback (most recent call last):
File "E:\SillyTavern\text-generation-webui\modules\ui_model_menu.py", line 245, in load_model_wrapper
shared.model, shared.tokenizer = load_model(selected_model, loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\SillyTavern\text-generation-webui\modules\models.py", line 94, in load_model
output = load_func_maploader
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\SillyTavern\text-generation-webui\modules\models.py", line 272, in llamacpp_loader
model, tokenizer = LlamaCppModel.from_pretrained(model_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\SillyTavern\text-generation-webui\modules\llamacpp_model.py", line 103, in from_pretrained
result.model = Llama(**params)
^^^^^^^^^^^^^^^
File "E:\SillyTavern\text-generation-webui\installer_files\env\Lib\site-packages\llama_cpp_cuda_tensorcores\llama.py", line 323, in init
self._model = _LlamaModel(
^^^^^^^^^^^^
File "E:\SillyTavern\text-generation-webui\installer_files\env\Lib\site-packages\llama_cpp_cuda_tensorcores_internals.py", line 55, in init
raise ValueError(f"Failed to load model from file: {path_model}")
ValueError: Failed to load model from file: models\gemma-2-27b-it-Q3_K_M.gguf
You probably need to update text gen to latest release
I updated text gen and am getting a new error with every model I try to load, not only with gemma-2
Traceback (most recent call last):
File "E:\SillyTavern\text-generation-webui\modules\ui_model_menu.py", line 246, in load_model_wrapper
shared.model, shared.tokenizer = load_model(selected_model, loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\SillyTavern\text-generation-webui\modules\models.py", line 94, in load_model
output = load_func_maploader
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\SillyTavern\text-generation-webui\modules\models.py", line 275, in llamacpp_loader
model, tokenizer = LlamaCppModel.from_pretrained(model_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\SillyTavern\text-generation-webui\modules\llamacpp_model.py", line 85, in from_pretrained
result.model = Llama(**params)
^^^^^^^^^^^^^^^
File "E:\SillyTavern\text-generation-webui\installer_files\env\Lib\site-packages\llama_cpp_cuda\llama.py", line 358, in init
self._model = self._stack.enter_context(contextlib.closing(_LlamaModel(
^^^^^^^^^^^^
File "E:\SillyTavern\text-generation-webui\installer_files\env\Lib\site-packages\llama_cpp_cuda_internals.py", line 49, in init
self.model = llama_cpp.llama_load_model_from_file(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [WinError -1073741795] Windows Error 0xc000001d