SageMaker Deployment Not Working (image missing timm library)
After the endpoint is deployed to SageMaker, the following error is received upon invoking the model:
ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{
"code": 400,
"type": "InternalServerException",
"message": "\nDetrForObjectDetection requires the timm library but it was not found in your environment. You can install it with pip:\npip install timm. Please note that you may need to restart your runtime after installation.\n"}
Looks like an external dependency is missing on the image
Hi @orcaman2 ,
It's now possible to use the DETR models without relying on the "timm" dependency. Just pass in revision=“no_timm” to the from_pretrained method:
from transformers import DetrForObjectDetection
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50", revision="no_timm")
when i ran this code to create an endpoint and tested the endpoint I got the same error.
import sagemaker
import boto3
from sagemaker.huggingface import HuggingFaceModel
try:
role = sagemaker.get_execution_role()
except ValueError:
iam = boto3.client('iam')
role = iam.get_role(RoleName='sagemaker_execution_role')['Role']['Arn']
Hub Model configuration. https://huggingface.co/models
hub = {
'HF_MODEL_ID':'facebook/detr-resnet-50',
'HF_TASK':'object-detection'
}
create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
transformers_version='4.37.0',
pytorch_version='2.1.0',
py_version='py310',
env=hub,
role=role,
)
deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
initial_instance_count=1, # number of instances
instance_type='ml.m5.xlarge' # ec2 instance type
)
@nielsr thank you for your help! could you please let me know how I can find no_timm revision?Thanks
cc @philschmid is there a way to pass a revision when deploying a model to SageMaker?
From this blog post: https://huggingface.co/blog/gptj-sagemaker, it looks like what you could do is:
- load the model using the Transformers library, passing the
revision - save the model using
torch.save - proceed as explained in the blog above