Error when deleting and Inference Endpoint in Sagemaker


I deploy a Mistral 7b model using AWS Sagemaker Jump start. I deleted model, endpoint config but when delete endpoint it is an error doesn't allow me to delete inference endpoint. It says " Cannot delete endpoint with inference component associated. Please delete inference component and try it again." I searched but I can not find any solution for this problem, please help me. the instance I deploy is g5.4xlarge and I have to pay $70/day for this problem.

Enter image description here

asked 3 months ago999 views
2 Answers
Accepted Answer


You are currently experiencing an issue where endpoints created using SageMaker Jump Start are not deleted.

The Inference component that is connected to the endpoint remains, and it occurs this issue.

You can resolve this issue in the following order through AWS CLI:

  1. (if necessary) AWS CLI Update

[+] Install or update the latest version of AWS CLI

  1. List and Delete the Endpoint Configurations
aws sagemaker list-endpoint-configs
aws sagemaker delete-endpoint-config --endpoint-config-name <config_name>
  1. List the Endpoints
aws sagemaker list-endpoints
  1. List the inference components
aws sagemaker list-inference-components

If you want to filter only the endpoint-associated inference component, you can use the option like below:

aws sagemaker list-inference-components --endpoint-name <endpoint_name>
  1. Delete the inference component
aws sagemaker delete-inference-component <component_name>

It will take a minute or two to completely delete it.

You can check whether it is deleted with the command in step4 (list the inference component).

  1. Delete the endpoint
aws sagemaker delete-endpoint --endpoint-name <endpoint_name>

Plus, you can also use boto3 to follow these steps.

If I have missed anything or answered wrong, please feel free to ask me again. Also you have any questions, comment please!

profile picture
answered 3 months ago
  • Thank you very much. You saved my day.

  • Point 5 is missing the --inference-component-name flag. Should be aws sagemaker delete-inference-component --inference-component-name <component_name>


Hello ! I'm trying to delete an inference component. But everytime I'm trying to delete the inference component after 2-3 minutes, when I check again I receive failure.

{ "InferenceComponents": [ { "CreationTime": "2024-02-03T19:15:04.464000+02:00", "InferenceComponentArn": "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/dev-epostbox-v1-summarization-inference-0", "InferenceComponentName": "dev-epostbox-v1-summarization-inference-0", "EndpointArn": "arn:aws:sagemaker:eu-central-1:904977793552:endpoint/dev-epostbox-v1-summarization-model", "EndpointName": "dev-epostbox-v1-summarization-model", "VariantName": "AllTraffic", "InferenceComponentStatus": "Failed", "LastModifiedTime": "2024-02-04T08:46:45.171000+02:00" } ] }

How should I proceed? Thanks, Alex

answered 18 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions