Hugging Face's logo

Felladrin
/
gguf-flan-t5-large

GGUF
Model card Files Community
1

Thanks, How ?

#1
by bharatcoder - opened
Discussion
bharatcoder

Thanks a lot for these!
If you do not mind, can you tell me how you converted these?
When I run the convert_hf script I am getting this error: raise ValueError(f"Mismatch between weight map and model parts for tensor names: {sym_diff}")
ValueError: Mismatch between weight map and model parts for tensor names: {'encoder.embed_tokens.weight', 'decoder.embed_tokens.weight'}

Felladrin
 Owner

Hi, @bharatcoder !

The T5 and Flan-T5 support is pretty recent, so my first thought is that it may need an update on llama.cpp and recompilation. Because for me the conversion was straightforward, using the code from main branch.

Here is the ipynb script:

!pip install huggingface_hub

from huggingface_hub import snapshot_download

snapshot_download(repo_id="google/flan-t5-large", local_dir="model", local_dir_use_symlinks=False, revision="main")

!git clone https://github.com/ggerganov/llama.cpp.git

%cd llama.cpp

!make

!pip install -r requirements.txt

%cd -

!python llama.cpp/convert_hf_to_gguf.py "model" --outtype f32

!llama.cpp/llama-quantize --output-tensor-type f16 --token-embedding-type f16 ./model/ggml-model-f32.gguf ./quantized/flan-t5-large.Q4_K_M.gguf Q4_K_M

Could you try it?

bharatcoder

Thanks for the steps! Is it necessary to download all the models? Or just the safetensors should be fine? Maybe I miss something? Yes, I have the latest llama.cpp and a clean and make.

Felladrin
 Owner

Ah, good quesition. In my case, I downloaded the snapshot including all files, so I'm not sure if it used safetensor or the pytorch model.

But I guess only those files are needed for converting:

  • config.json
  • generation_config.json
  • model.safetensors
  • special_tokens_map.json
  • spiece.model
  • tokenizer.json
  • tokenizer_config.json
bharatcoder

Yes, true, It only needs the safetensors, I think. I get the following error, even with your command:
llama.cpp % python convert_hf_to_gguf.py model --outtype f32 --outfile model/f32.gguf
INFO:hf-to-gguf:Loading model: model
INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
INFO:hf-to-gguf:Set model parameters
INFO:hf-to-gguf:Set model tokenizer
INFO:gguf.vocab:Setting special token type eos to 1
INFO:gguf.vocab:Setting special token type unk to 2
INFO:gguf.vocab:Setting special token type pad to 0
INFO:hf-to-gguf:Exporting model...
INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
INFO:hf-to-gguf:gguf: loading model part 'model-00001-of-00002.safetensors'
INFO:hf-to-gguf:dec.blk.0.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.0.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.0.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.0.attn_rel_b.weight, torch.float32 --> F32, shape = {32, 32}
INFO:hf-to-gguf:dec.blk.0.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.0.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.0.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.0.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.0.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.0.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.0.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.0.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.0.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.0.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.0.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.1.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.1.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.1.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.1.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.1.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.1.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.1.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.1.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.1.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.1.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.1.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.1.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.1.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.1.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.10.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.10.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.10.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.10.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.10.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.10.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.10.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.10.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.10.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.10.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.10.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.10.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.10.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.10.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.11.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.11.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.11.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.11.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.11.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.11.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.11.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.11.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.11.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.11.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.11.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.11.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.11.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.11.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.12.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.12.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.12.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.12.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.12.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.12.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.12.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.12.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.12.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.12.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.12.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.12.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.12.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.12.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.13.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.13.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.13.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.13.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.13.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.13.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.13.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.13.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.13.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.13.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.13.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.13.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.13.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.13.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.14.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.14.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.14.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.14.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.14.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.14.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.14.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.14.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.14.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.14.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.14.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.14.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.14.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.14.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.15.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.15.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.15.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.15.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.15.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.15.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.15.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.15.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.15.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.15.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.15.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.15.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.15.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.15.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.16.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.16.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.16.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.16.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.16.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.16.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.16.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.16.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.16.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.16.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.16.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.16.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.16.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.16.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.17.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.17.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.17.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.17.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.17.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.17.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.17.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.17.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.17.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.17.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.2.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.2.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.2.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.2.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.2.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.2.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.2.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.2.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.2.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.2.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.2.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.2.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.2.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.2.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.3.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.3.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.3.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.3.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.3.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.3.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.3.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.3.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.3.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.3.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.3.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.3.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.3.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.3.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.4.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.4.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.4.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.4.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.4.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.4.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.4.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.4.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.4.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.4.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.4.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.4.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.4.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.4.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.5.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.5.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.5.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.5.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.5.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.5.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.5.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.5.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.5.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.5.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.5.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.5.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.5.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.5.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.6.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.6.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.6.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.6.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.6.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.6.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.6.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.6.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.6.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.6.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.6.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.6.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.6.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.6.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.7.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.7.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.7.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.7.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.7.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.7.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.7.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.7.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.7.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.7.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.7.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.7.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.7.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.7.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.8.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.8.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.8.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.8.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.8.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.8.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.8.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.8.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.8.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.8.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.8.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.8.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.8.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.8.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.9.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.9.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.9.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.9.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.9.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.9.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.9.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.9.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.9.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.9.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.9.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.9.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.9.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.9.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.0.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.0.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.0.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.0.attn_rel_b.weight, torch.float32 --> F32, shape = {32, 32}
INFO:hf-to-gguf:enc.blk.0.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.0.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.0.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.0.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.0.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.0.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.1.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.1.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.1.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.1.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.1.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.1.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.1.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.1.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.1.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.10.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.10.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.10.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.10.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.10.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.10.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.10.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.10.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.10.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.11.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.11.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.11.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.11.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.11.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.11.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.11.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.11.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.11.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.12.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.12.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.12.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.12.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.12.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.12.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.12.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.12.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.12.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.13.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.13.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.13.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.13.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.13.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.13.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.13.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.13.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.13.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.14.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.14.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.14.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.14.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.14.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.14.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.14.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.14.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.14.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.15.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.15.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.15.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.15.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.15.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.15.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.15.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.15.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.15.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.16.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.16.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.16.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.16.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.16.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.16.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.16.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.16.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.16.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.17.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.17.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.17.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.17.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.17.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.17.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.17.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.17.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.17.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.18.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.18.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.18.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.18.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.18.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.18.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.18.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.18.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.18.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.19.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.19.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.19.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.19.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.19.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.19.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.19.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.19.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.19.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.2.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.2.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.2.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.2.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.2.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.2.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.2.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.2.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.2.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.20.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.20.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.20.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.20.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.20.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.20.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.20.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.20.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.20.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.21.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.21.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.21.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.21.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.21.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.21.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.21.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.21.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.21.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.22.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.22.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.22.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.22.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.22.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.22.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.22.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.22.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.22.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.23.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.23.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.23.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.23.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.23.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.23.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.23.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.23.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.23.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.3.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.3.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.3.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.3.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.3.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.3.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.3.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.3.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.3.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.4.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.4.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.4.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.4.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.4.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.4.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.4.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.4.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.4.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.5.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.5.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.5.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.5.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.5.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.5.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.5.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.5.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.5.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.6.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.6.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.6.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.6.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.6.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.6.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.6.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.6.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.6.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.7.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.7.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.7.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.7.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.7.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.7.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.7.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.7.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.7.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.8.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.8.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.8.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.8.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.8.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.8.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.8.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.8.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.8.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.9.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.9.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.9.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.9.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:enc.blk.9.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.blk.9.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.9.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:enc.blk.9.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:enc.blk.9.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:enc.output_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:token_embd.weight, torch.float32 --> F32, shape = {2048, 32128}
INFO:hf-to-gguf:gguf: loading model part 'model-00002-of-00002.safetensors'
INFO:hf-to-gguf:dec.blk.17.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.17.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.17.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.17.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.18.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.18.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.18.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.18.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.18.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.18.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.18.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.18.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.18.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.18.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.18.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.18.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.18.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.18.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.19.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.19.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.19.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.19.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.19.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.19.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.19.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.19.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.19.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.19.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.19.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.19.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.19.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.19.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.20.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.20.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.20.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.20.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.20.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.20.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.20.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.20.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.20.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.20.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.20.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.20.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.20.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.20.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.21.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.21.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.21.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.21.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.21.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.21.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.21.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.21.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.21.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.21.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.21.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.21.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.21.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.21.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.22.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.22.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.22.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.22.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.22.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.22.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.22.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.22.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.22.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.22.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.22.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.22.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.22.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.22.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.23.attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.23.attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.23.attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.23.attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.23.attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.23.cross_attn_k.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.23.cross_attn_o.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.23.cross_attn_q.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.23.cross_attn_v.weight, torch.float32 --> F32, shape = {2048, 2048}
INFO:hf-to-gguf:dec.blk.23.cross_attn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.blk.23.ffn_gate.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.23.ffn_up.weight, torch.float32 --> F32, shape = {2048, 5120}
INFO:hf-to-gguf:dec.blk.23.ffn_down.weight, torch.float32 --> F32, shape = {5120, 2048}
INFO:hf-to-gguf:dec.blk.23.ffn_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:dec.output_norm.weight, torch.float32 --> F32, shape = {2048}
INFO:hf-to-gguf:output.weight, torch.float32 --> F32, shape = {2048, 32128}
Traceback (most recent call last):
File "/Volumes/d/apps/llama.cpp/llama.cpp/convert_hf_to_gguf.py", line 3361, in
main()
File "/Volumes/d/apps/llama.cpp/llama.cpp/convert_hf_to_gguf.py", line 3355, in main
model_instance.write()
File "/Volumes/d/apps/llama.cpp/llama.cpp/convert_hf_to_gguf.py", line 330, in write
self.write_tensors()
File "/Volumes/d/apps/llama.cpp/llama.cpp/convert_hf_to_gguf.py", line 249, in write_tensors
for name, data_torch in self.get_tensors():
File "/Volumes/d/apps/llama.cpp/llama.cpp/convert_hf_to_gguf.py", line 158, in get_tensors
raise ValueError(f"Mismatch between weight map and model parts for tensor names: {sym_diff}")
ValueError: Mismatch between weight map and model parts for tensor names: {'decoder.embed_tokens.weight', 'encoder.embed_tokens.weight'}

Felladrin
 Owner

Hmm, I'm out of ideas.

To confirm, I've just run the script I posted above, and the conversion worked (I see it also used the safetensors file)

INFO:hf-to-gguf:Loading model: model
INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
INFO:hf-to-gguf:Set model parameters
INFO:hf-to-gguf:Set model tokenizer
INFO:gguf.vocab:Setting special token type eos to 1
INFO:gguf.vocab:Setting special token type unk to 2
INFO:gguf.vocab:Setting special token type pad to 0
INFO:hf-to-gguf:Exporting model...
INFO:hf-to-gguf:gguf: loading model part 'model.safetensors'
INFO:hf-to-gguf:dec.blk.0.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.0.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.0.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.0.attn_rel_b.weight,        torch.float32 --> F32, shape = {16, 32}
INFO:hf-to-gguf:dec.blk.0.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.0.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.0.cross_attn_k.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.0.cross_attn_o.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.0.cross_attn_q.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.0.cross_attn_v.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.0.cross_attn_norm.weight,   torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.0.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.0.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.0.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.0.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.1.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.1.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.1.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.1.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.1.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.1.cross_attn_k.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.1.cross_attn_o.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.1.cross_attn_q.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.1.cross_attn_v.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.1.cross_attn_norm.weight,   torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.1.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.1.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.1.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.1.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.10.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.10.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.10.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.10.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.10.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.10.cross_attn_k.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.10.cross_attn_o.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.10.cross_attn_q.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.10.cross_attn_v.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.10.cross_attn_norm.weight,  torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.10.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.10.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.10.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.10.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.11.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.11.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.11.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.11.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.11.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.11.cross_attn_k.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.11.cross_attn_o.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.11.cross_attn_q.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.11.cross_attn_v.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.11.cross_attn_norm.weight,  torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.11.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.11.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.11.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.11.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.12.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.12.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.12.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.12.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.12.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.12.cross_attn_k.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.12.cross_attn_o.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.12.cross_attn_q.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.12.cross_attn_v.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.12.cross_attn_norm.weight,  torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.12.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.12.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.12.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.12.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.13.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.13.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.13.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.13.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.13.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.13.cross_attn_k.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.13.cross_attn_o.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.13.cross_attn_q.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.13.cross_attn_v.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.13.cross_attn_norm.weight,  torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.13.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.13.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.13.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.13.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.14.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.14.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.14.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.14.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.14.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.14.cross_attn_k.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.14.cross_attn_o.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.14.cross_attn_q.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.14.cross_attn_v.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.14.cross_attn_norm.weight,  torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.14.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.14.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.14.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.14.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.15.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.15.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.15.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.15.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.15.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.15.cross_attn_k.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.15.cross_attn_o.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.15.cross_attn_q.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.15.cross_attn_v.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.15.cross_attn_norm.weight,  torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.15.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.15.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.15.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.15.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.16.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.16.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.16.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.16.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.16.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.16.cross_attn_k.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.16.cross_attn_o.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.16.cross_attn_q.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.16.cross_attn_v.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.16.cross_attn_norm.weight,  torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.16.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.16.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.16.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.16.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.17.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.17.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.17.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.17.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.17.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.17.cross_attn_k.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.17.cross_attn_o.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.17.cross_attn_q.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.17.cross_attn_v.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.17.cross_attn_norm.weight,  torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.17.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.17.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.17.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.17.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.18.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.18.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.18.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.18.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.18.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.18.cross_attn_k.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.18.cross_attn_o.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.18.cross_attn_q.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.18.cross_attn_v.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.18.cross_attn_norm.weight,  torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.18.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.18.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.18.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.18.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.19.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.19.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.19.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.19.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.19.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.19.cross_attn_k.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.19.cross_attn_o.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.19.cross_attn_q.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.19.cross_attn_v.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.19.cross_attn_norm.weight,  torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.19.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.19.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.19.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.19.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.2.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.2.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.2.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.2.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.2.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.2.cross_attn_k.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.2.cross_attn_o.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.2.cross_attn_q.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.2.cross_attn_v.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.2.cross_attn_norm.weight,   torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.2.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.2.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.2.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.2.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.20.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.20.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.20.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.20.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.20.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.20.cross_attn_k.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.20.cross_attn_o.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.20.cross_attn_q.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.20.cross_attn_v.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.20.cross_attn_norm.weight,  torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.20.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.20.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.20.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.20.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.21.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.21.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.21.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.21.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.21.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.21.cross_attn_k.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.21.cross_attn_o.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.21.cross_attn_q.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.21.cross_attn_v.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.21.cross_attn_norm.weight,  torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.21.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.21.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.21.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.21.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.22.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.22.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.22.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.22.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.22.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.22.cross_attn_k.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.22.cross_attn_o.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.22.cross_attn_q.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.22.cross_attn_v.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.22.cross_attn_norm.weight,  torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.22.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.22.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.22.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.22.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.23.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.23.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.23.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.23.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.23.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.23.cross_attn_k.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.23.cross_attn_o.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.23.cross_attn_q.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.23.cross_attn_v.weight,     torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.23.cross_attn_norm.weight,  torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.23.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.23.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.23.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.23.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.3.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.3.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.3.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.3.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.3.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.3.cross_attn_k.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.3.cross_attn_o.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.3.cross_attn_q.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.3.cross_attn_v.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.3.cross_attn_norm.weight,   torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.3.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.3.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.3.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.3.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.4.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.4.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.4.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.4.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.4.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.4.cross_attn_k.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.4.cross_attn_o.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.4.cross_attn_q.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.4.cross_attn_v.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.4.cross_attn_norm.weight,   torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.4.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.4.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.4.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.4.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.5.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.5.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.5.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.5.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.5.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.5.cross_attn_k.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.5.cross_attn_o.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.5.cross_attn_q.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.5.cross_attn_v.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.5.cross_attn_norm.weight,   torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.5.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.5.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.5.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.5.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.6.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.6.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.6.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.6.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.6.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.6.cross_attn_k.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.6.cross_attn_o.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.6.cross_attn_q.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.6.cross_attn_v.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.6.cross_attn_norm.weight,   torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.6.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.6.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.6.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.6.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.7.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.7.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.7.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.7.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.7.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.7.cross_attn_k.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.7.cross_attn_o.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.7.cross_attn_q.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.7.cross_attn_v.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.7.cross_attn_norm.weight,   torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.7.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.7.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.7.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.7.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.8.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.8.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.8.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.8.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.8.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.8.cross_attn_k.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.8.cross_attn_o.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.8.cross_attn_q.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.8.cross_attn_v.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.8.cross_attn_norm.weight,   torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.8.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.8.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.8.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.8.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.9.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.9.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.9.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.9.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.9.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.9.cross_attn_k.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.9.cross_attn_o.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.9.cross_attn_q.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.9.cross_attn_v.weight,      torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:dec.blk.9.cross_attn_norm.weight,   torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.blk.9.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.9.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:dec.blk.9.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:dec.blk.9.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:dec.output_norm.weight,             torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.0.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.0.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.0.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.0.attn_rel_b.weight,        torch.float32 --> F32, shape = {16, 32}
INFO:hf-to-gguf:enc.blk.0.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.0.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.0.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.0.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.0.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.0.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.1.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.1.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.1.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.1.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.1.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.1.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.1.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.1.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.1.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.10.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.10.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.10.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.10.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.10.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.10.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.10.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.10.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.10.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.11.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.11.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.11.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.11.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.11.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.11.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.11.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.11.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.11.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.12.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.12.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.12.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.12.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.12.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.12.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.12.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.12.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.12.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.13.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.13.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.13.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.13.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.13.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.13.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.13.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.13.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.13.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.14.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.14.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.14.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.14.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.14.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.14.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.14.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.14.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.14.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.15.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.15.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.15.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.15.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.15.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.15.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.15.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.15.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.15.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.16.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.16.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.16.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.16.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.16.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.16.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.16.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.16.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.16.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.17.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.17.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.17.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.17.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.17.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.17.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.17.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.17.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.17.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.18.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.18.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.18.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.18.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.18.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.18.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.18.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.18.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.18.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.19.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.19.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.19.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.19.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.19.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.19.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.19.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.19.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.19.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.2.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.2.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.2.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.2.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.2.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.2.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.2.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.2.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.2.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.20.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.20.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.20.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.20.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.20.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.20.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.20.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.20.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.20.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.21.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.21.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.21.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.21.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.21.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.21.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.21.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.21.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.21.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.22.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.22.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.22.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.22.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.22.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.22.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.22.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.22.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.22.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.23.attn_k.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.23.attn_o.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.23.attn_q.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.23.attn_v.weight,           torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.23.attn_norm.weight,        torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.23.ffn_gate.weight,         torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.23.ffn_up.weight,           torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.23.ffn_down.weight,         torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.23.ffn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.3.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.3.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.3.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.3.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.3.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.3.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.3.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.3.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.3.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.4.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.4.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.4.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.4.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.4.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.4.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.4.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.4.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.4.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.5.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.5.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.5.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.5.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.5.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.5.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.5.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.5.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.5.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.6.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.6.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.6.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.6.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.6.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.6.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.6.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.6.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.6.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.7.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.7.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.7.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.7.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.7.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.7.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.7.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.7.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.7.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.8.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.8.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.8.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.8.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.8.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.8.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.8.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.8.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.8.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.9.attn_k.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.9.attn_o.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.9.attn_q.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.9.attn_v.weight,            torch.float32 --> F32, shape = {1024, 1024}
INFO:hf-to-gguf:enc.blk.9.attn_norm.weight,         torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.blk.9.ffn_gate.weight,          torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.9.ffn_up.weight,            torch.float32 --> F32, shape = {1024, 2816}
INFO:hf-to-gguf:enc.blk.9.ffn_down.weight,          torch.float32 --> F32, shape = {2816, 1024}
INFO:hf-to-gguf:enc.blk.9.ffn_norm.weight,          torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:enc.output_norm.weight,             torch.float32 --> F32, shape = {1024}
INFO:hf-to-gguf:output.weight,                      torch.float32 --> F32, shape = {1024, 32128}
INFO:hf-to-gguf:token_embd.weight,                  torch.float32 --> F32, shape = {1024, 32128}
INFO:gguf.gguf_writer:Writing the following files:
INFO:gguf.gguf_writer:model/ggml-model-f32.gguf: n_tensors = 558, total_size = 3.1G
Writing: 100% 3.13G/3.13G [00:36<00:00, 85.9Mbyte/s]
INFO:hf-to-gguf:Model successfully exported to model/ggml-model-f32.gguf
Felladrin
 Owner

Ah, I see on your logs there's

INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
INFO:hf-to-gguf:gguf: loading model part 'model-00001-of-00002.safetensors'

So you're converting some other T5/Flan-T5 model, I believe. In this case, it might be an issue with the model (or it's some slight different model architecture).
I think it is worth creating an issue on llama.cpp repo about it.

bharatcoder

Yes, you right! Thanks. I am doing this one - google/flan-t5-xl

Felladrin changed discussion status to closed

Sign up or log in to comment