Getting "KeyError: 'phi3'"" while trying to run using transformers
Is there a fix for this? I am on the latest transformers library version: 4.40.2
{
"name": "KeyError",
"message": "'phi3'",
"stack": "---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
Cell In[2], line 1
----> 1 model = AutoModelForVision2Seq.from_pretrained(model_name_or_path, trust_remote_code=True)
File /home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:523, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
520 if kwargs.get(\"quantization_config\", None) is not None:
521 _ = kwargs.pop(\"quantization_config\")
--> 523 config, kwargs = AutoConfig.from_pretrained(
524 pretrained_model_name_or_path,
525 return_unused_kwargs=True,
526 trust_remote_code=trust_remote_code,
527 code_revision=code_revision,
528 _commit_hash=commit_hash,
529 **hub_kwargs,
530 **kwargs,
531 )
533 # if torch_dtype=auto was passed here, ensure to pass it on
534 if kwargs_orig.get(\"torch_dtype\", None) == \"auto\":
File /home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:942, in AutoConfig.from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
940 if os.path.isdir(pretrained_model_name_or_path):
941 config_class.register_for_auto_class()
--> 942 return config_class.from_pretrained(pretrained_model_name_or_path, **kwargs)
943 elif \"model_type\" in config_dict:
944 try:
File /home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/transformers/configuration_utils.py:609, in PretrainedConfig.from_pretrained(cls, pretrained_model_name_or_path, cache_dir, force_download, local_files_only, token, revision, **kwargs)
603 if \"model_type\" in config_dict and hasattr(cls, \"model_type\") and config_dict[\"model_type\"] != cls.model_type:
604 logger.warning(
605 f\"You are using a model of type {config_dict['model_type']} to instantiate a model of type \"
606 f\"{cls.model_type}. This is not supported for all configurations of models and can yield errors.\"
607 )
--> 609 return cls.from_dict(config_dict, **kwargs)
File /home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/transformers/configuration_utils.py:761, in PretrainedConfig.from_dict(cls, config_dict, **kwargs)
758 # We remove it from kwargs so that it does not appear in `return_unused_kwargs`.
759 config_dict[\"attn_implementation\"] = kwargs.pop(\"attn_implementation\", None)
--> 761 config = cls(**config_dict)
763 if hasattr(config, \"pruned_heads\"):
764 config.pruned_heads = {int(key): value for key, value in config.pruned_heads.items()}
File /home/zeus/.cache/huggingface/modules/transformers_modules/Salesforce/blip3-phi3-mini-instruct-r-v1/0e928237baf450032f063e295be64238fc9d6fd5/configuration_blip_3.py:139, in Blip3Config.__init__(self, vision_encoder_config, vision_tokenizer_config, text_config, **kwargs)
136 self.vision_tokenizer_config = Blip3VisionTokenizerConfig(**vision_tokenizer_config)
138 text_model_type = text_config[\"model_type\"] if \"model_type\" in text_config else \"phi3\"
--> 139 self.text_config = CONFIG_MAPPING[text_model_type](**text_config)
141 for key in ['initial_tokenizer_len', 'pad_token_id']:
142 if key not in self.text_config.to_dict():
File /home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:647, in _LazyConfigMapping.__getitem__(self, key)
645 return self._extra_content[key]
646 if key not in self._mapping:
--> 647 raise KeyError(key)
648 value = self._mapping[key]
649 module_name = model_type_to_module_name(key)
KeyError: 'phi3'"
}
if it still gives you an error, you can try to modify your configuration_auto.py from the one available on github
https://github.com/huggingface/transformers/blob/main/src/transformers/models/auto/configuration_auto.py
and eventually add the phi3 3 folder
https://github.com/huggingface/transformers/tree/main/src/transformers/models/phi3
it's not a good idea to modify the installed library, but you can use this workaround until a new version of transformers is released
Moreover, be sure that your IDE python interpreter is correctly set up
Hi @dhruv-anand-aintech , thanks for your interest. Please note that in our readme file
We require the use of the development version ("4.41.0.dev0") of the transformers library. To get it, as of 05/07/2024, one can use pip uninstall -y transformers && pip install git+https://github.com/huggingface/transformers.
I think that phi3 in not available in 4.40.2. Please let us know if you have more questions.
You should install the Transformers library like this.
!pip install git+https://github.com/huggingface/transformers
Thanks. That resolves the initial error.
Now I run out of VRAM on a single T4:OutOfMemoryError: CUDA out of memory. Tried to allocate 192.00 MiB. GPU 0 has a total capacity of 14.75 GiB of which 135.06 MiB is free. Process 20299 has 14.61 GiB memory in use. Of the allocated memory 14.42 GiB is allocated by PyTorch, and 97.78 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
What is the recommended hardware to use for this model?
Also, can it be hosted on a platform like Replicate?
I see https://replicate.com/zsxkib/blip-3?prediction=ev9ab62nw9rg80cfe7krsrx66r, but it is not warm, and I'm not sure it has the latest updates made to the model setup.
Seems like you need about 1GB per k of context plus space for the parameters of 2 bytes per param, plus some room to manage all of it. That's why it's trying to allocate 192GB of memory. Full 128k context probably isn't runnable on that hardware you'd probably max that out with 16k context.