Unable to load the model
Application: LM Studio 0.3.3 ( https://lmstudio.ai/blog/lmstudio-v0.3.3 )
Error log:
🥲 Failed to load the model
Failed to load model
llama.cpp error: 'error loading model architecture: unknown model architecture: '''
Other models(like gemma-2b) work ok
What I'm doing wrong?
Hi, @xardbaiz Flan model belongs to the encoder-decoder Language model. That architecture was not supported yet last year.
LMStudio use llama.cpp under the hood. llama.cpp was designed for decoder-only architecture. But later, it has been added.
FYI, I created the gguf file using tensor-tools from candle.
conversion source
I think t5 model has been added to llama.cpp, and in order to be supported, it needs to be converted by llama.cpp. This is -> How To
Hey, thanks for the fast reply!
Yeah. candle also doesn't work on llama.cpp :( Just tried
On Windows it's not so easy to play with all that python libs
Have you seen any already converted flan t5 GGUF models (I'm interested in t5 XL)?
From my knowledge, I have no idea. I can do it for you, but right now, I'm in a small city with bad internet speed. I will be back in the town only on the weekends.
Cool!! No rush, but it would be soooo awesome to have this model in GGUF! So it waits couple weaks
Thanks again !