mlx-community

/

CodeLlama-70b-hf-4bit-MLX

like 12

Hi there, hello. I tried the provided example, and got the following error:

Traceback (most recent call last):
  File "project/app.py", line 3, in <module>
    model, tokenizer = load("mlx-community/CodeLlama-70b-hf-4bit-MLX")
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/mlx_lm/utils.py", line 279, in load
    model = load_model(model_path)
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/mlx_lm/utils.py", line 220, in load_model
    weights.update(mx.load(wf))
                   ^^^^^^^^^^^
RuntimeError: [load] Invalid header in file ~/.cache/huggingface/hub/models--mlx-community--CodeLlama-70b-hf-4bit-MLX/snapshots/2fd732eb3a2685a22a03258b8191af356566c203/model-00004-of-00008.safetensors

I redownloaded the files multiple times with the same result. I also tried to download them by hands from Files pages using browser and manually put the files into the cache folder. But same error.

What am I doing wrong?