Trouble Importing Model from Transformers Library
I updated my transformers library using the command pip install git+https://github.com/huggingface/transformers. However, I am still encountering an ImportError when trying to load a specific model. Here is the code I used and the error message I received:
```
ImportError Traceback (most recent call last)
Cell In[2], line 2
1 # Load model directly
----> 2 from transformers import LlavaPhiWithMaskDecoderForInstanceAfterPoolingRandom
3 model = LlavaPhiWithMaskDecoderForInstanceAfterPoolingRandom.from_pretrained("EnmingZhang/PSALM")
ImportError: cannot import name 'LlavaPhiWithMaskDecoderForInstanceAfterPoolingRandom' from 'transformers' (/opt/conda/lib/python3.10/site-packages/transformers/init.py)
Is LlavaPhiWithMaskDecoderForInstanceAfterPoolingRandom correct?
when I try:
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("EnmingZhang/PSALM")
I get:
ValueError: The checkpoint you are trying to load has model type `llava_phi` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
I'm in psalm/train I found this sentence model = PSALM.from_pretrained(...) in /train.py. Then I tried to run this code from psalm.model.language_model.llava_phi import PSALM and got an error:
```
KeyError Traceback (most recent call last)
Cell In[7], line 1
----> 1 from psalm.model.language_model.llava_phi import PSALM
File :1027, in find_and_load(name, import)
File :1002, in find_and_load_unlocked(name, import)
File :945, in _find_spec(name, path, target)
File :1439, in find_spec(cls, fullname, path, target)
File :1405, in _get_spec(cls, fullname, path, target)
File :1255, in iter(self)
File :1242, in _recalculate(self)
File :1238, in _get_parent_path(self)
KeyError: 'psalm.model'
How should I load this model in python?
Hi
@linshi1111
The name of LlavaPhiWithMaskDecoderForInstanceAfterPoolingRandom is deprecated, it call PSALM now. I'll modify this repo to remove this name.
And currently, PSALM can not support AutoModelForCausalLM loading and I'll work on it and will ping you when it's fine.
If you want to using PSALM, like traing or evaluating, a proper way is following our official code https://github.com/zamling/PSALM and Installation guidelines.
gotcha. Thanks for open sourcing.
Hi @linshi1111
The name ofLlavaPhiWithMaskDecoderForInstanceAfterPoolingRandomis deprecated, it call PSALM now. I'll modify this repo to remove this name.
And currently, PSALM can not support AutoModelForCausalLM loading and I'll work on it and will ping you when it's fine.
If you want to using PSALM, like traing or evaluating, a proper way is following our official code https://github.com/zamling/PSALM and Installation guidelines.
Do you mean that the trained model on hugging face cannot be used right now and should be deployed step by step following the training steps on github, but this takes a lot of time?
I found the usage of LlavaPhiWithMaskDecoderForInstanceAfterPoolingRandom in the model usage guide on the Hugging Face page. Specifically, this can be found by clicking the "Use this model" button and selecting the "Transformers" library, where the following code example is provided:
from transformers import LlavaPhiWithMaskDecoderForInstanceAfterPoolingRandom # (need update instruction)
model = LlavaPhiWithMaskDecoderForInstanceAfterPoolingRandom.from_pretrained("EnmingZhang/PSALM")
Yes, I mean I want to quickly test if PSALM can be migrated to my research. Therefore, it might be necessary to update the instructions on how to load this pre-trained model.
Hi
@XiaoChenxixi
We only use huggingface to publish our model weights. In this model, there are so many other package, such as Detectron2, which can not merge into huggingfac easily.
So please follow our official code.
BTW, if you wanna use PSALM as a Demo tool (like deploy in Gradio), please give me some time and I do have a plan to do this. I will ping you guys when it is available
Thanks for your interest !
Thank you very much for your answer. One more question that I hope you can clarify is, can the model training weights downloaded here be used to directly replace the model paths in the evaluation? In addition, I do not seem to see the VQA code implementation process, this specific implementation is?