checkpoint you are trying to load has model type `llava_llama` but Transformers does not recognize this architecture
#1
by
joris-sense
- opened
The transformers library has version 4.42.4.
I am getting the following error when using the "use this model" code generated by transformers in Colab:
/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:99: UserWarning:
Error while fetching `HF_TOKEN` secret value from your vault: 'Requesting secret HF_TOKEN timed out. Secrets can only be fetched when running from the Colab UI.'.
You are not authenticated with the Hugging Face Hub in this notebook.
If the error persists, please let us know by opening an issue on GitHub (https://github.com/huggingface/huggingface_hub/issues/new).
warnings.warn(
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
992 try:
--> 993 config_class = CONFIG_MAPPING[config_dict["model_type"]]
994 except KeyError:
3 frames
KeyError: 'llava_llama'
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
993 config_class = CONFIG_MAPPING[config_dict["model_type"]]
994 except KeyError:
--> 995 raise ValueError(
996 f"The checkpoint you are trying to load has model type `{config_dict['model_type']}` "
997 "but Transformers does not recognize this architecture. This could be because of an "
ValueError: The checkpoint you are trying to load has model type `llava_llama` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
Hi
@joris-sense
,
to use the LLaVA-MORE model, you can run the script available at the following link: https://github.com/aimagelab/LLaVA-MORE/blob/main/llava/eval/run_llava.py
Additionally, you'll find more useful information in the repository!
The transformers version to use is '4.43.1'.
Hey, thanks! I had some trouble with that script as well, I created a PR documenting what I did/could have done.