Delete Inference Component Sagemaker

0

Hello ! I'm trying to delete some inference components from deleted endpoints. I was trying differents methods by aws cli or boto3.

aws sagemaker list-inference-components --region eu-central-1
{
    "InferenceComponents": [
        {
            "CreationTime": 1707619639.152,
            "InferenceComponentArn": "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/huggingface-pytorch-tgi-inference-2024-02-11-02-1707619638-aa40",
            "InferenceComponentName": "huggingface-pytorch-tgi-inference-2024-02-11-02-1707619638-aa40",
            "EndpointArn": "arn:aws:sagemaker:eu-central-1:904977793552:endpoint/dev-epostbox-v2-summarization-endpoint",
            "EndpointName": "dev-epostbox-v2-summarization-endpoint",
            "VariantName": "AllTraffic",
            "InferenceComponentStatus": "Failed",
            "LastModifiedTime": 1707620610.616
        },
        {
            "CreationTime": 1707539285.172,
            "InferenceComponentArn": "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/dev-epostbox-v1-summarization-inference-3",
            "InferenceComponentName": "dev-epostbox-v1-summarization-inference-3",
            "EndpointArn": "arn:aws:sagemaker:eu-central-1:904977793552:endpoint/dev-epostbox-v1-summarization-endpoint",
            "EndpointName": "dev-epostbox-v1-summarization-endpoint",
            "VariantName": "AllTraffic",
            "InferenceComponentStatus": "Failed",
            "LastModifiedTime": 1707548094.115
        },
        {
            "CreationTime": 1707075798.484,
            "InferenceComponentArn": "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/dev-epostbox-v1-summarization-inference-1",
            "InferenceComponentName": "dev-epostbox-v1-summarization-inference-1",
            "EndpointArn": "arn:aws:sagemaker:eu-central-1:904977793552:endpoint/dev-epostbox-v1-summarization-endpoint",
            "EndpointName": "dev-epostbox-v1-summarization-endpoint",
            "VariantName": "AllTraffic",
            "InferenceComponentStatus": "Failed",
            "LastModifiedTime": 1707081941.992
        },
        {
            "CreationTime": 1707023267.455,
            "InferenceComponentArn": "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/DEMO-1707023266-6b96-inference-component-0",
            "InferenceComponentName": "DEMO-1707023266-6b96-inference-component-0",
            "EndpointArn": "arn:aws:sagemaker:eu-central-1:904977793552:endpoint/demo-1707022974-d5b4-endpoint",
            "EndpointName": "DEMO-1707022974-d5b4-endpoint",
            "VariantName": "AllTraffic",
            "InferenceComponentStatus": "Failed",
            "LastModifiedTime": 1707029439.106
        },
        {
            "CreationTime": 1706980504.464,
            "InferenceComponentArn": "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/dev-epostbox-v1-summarization-inference-0",
            "InferenceComponentName": "dev-epostbox-v1-summarization-inference-0",
            "EndpointArn": "arn:aws:sagemaker:eu-central-1:904977793552:endpoint/dev-epostbox-v1-summarization-model",
            "EndpointName": "dev-epostbox-v1-summarization-model",
            "VariantName": "AllTraffic",
            "InferenceComponentStatus": "Failed",
            "LastModifiedTime": 1707031949.226
        },
        {
            "CreationTime": 1706788995.558,
            "InferenceComponentArn": "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/DEMO-1706788994-407e-inference-component-0",
            "InferenceComponentName": "DEMO-1706788994-407e-inference-component-0",
            "EndpointArn": "arn:aws:sagemaker:eu-central-1:904977793552:endpoint/demo-1706784905-e5b4-endpoint",
            "EndpointName": "DEMO-1706784905-e5b4-endpoint",
            "VariantName": "AllTraffic",
            "InferenceComponentStatus": "Failed",
            "LastModifiedTime": 1706800841.762
        }
    ]
}

When I'm trying to delete one of the inference components I receive the error

aws sagemaker delete-inference-component --inference-component-name dev-epostbox-v1-summarization-inference-0

An error occurred (ValidationException) when calling the DeleteInferenceComponent operation: Could not find endpoint "dev-epostbox-v1-summarization-model".

What should I do?

Alex
asked 3 months ago222 views
3 Answers
0

Hi,

Your objects seem to be inference endpoints.

So, can you try aws sagemaker delete-endpoint ?

See https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sagemaker/delete-endpoint.html

Best,

Didier

profile pictureAWS
EXPERT
answered 3 months ago
  • Hello and thanks for your time .The objects are inference-components (aws sagemaker list-inference-components --region eu-central-1) . I don't have any endpoints in service right now. That's why it's very strange that I cannot delete these objects.

0

Understood on your latest comment, Alex (no endpoint in service). Have you tried using the ARN instead?

aws sagemaker delete-inference-component --inference-component-name "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/dev-epostbox-v1-summarization-inference-0"

I would also ensure that you have the latest copy of the AWS CLI version 2

If the above solves the issue, please choose this as the Accepted Answer so others on re:Post may benefit - Thank you!

profile pictureAWS
answered 3 months ago
  • Yes, but inference-component-name expects: regular expression pattern: ^a-zA-Z0-9?$ Member must have length less than or equal to 63

0

Hi, Alex

Try the preliminary steps included below also.

  1. List and Delete the Endpoint Configurations
aws sagemaker list-endpoint-configs
aws sagemaker delete-endpoint-config --endpoint-config-name <config_name>
  1. List the Endpoints
aws sagemaker list-endpoints
  1. List the inference components
aws sagemaker list-inference-components
  1. If you want to filter only the endpoint-associated inference component, you can use the option like below:
aws sagemaker list-inference-components --endpoint-name <endpoint_name>
  1. Delete the inference component
aws sagemaker delete-inference-component --inference-component-name <component_name>

It will take a minute or two to completely delete it.

You can check whether it is deleted with the command in step4 (list the inference component).

  1. Delete the endpoint
aws sagemaker delete-endpoint --endpoint-name <endpoint_name>
profile pictureAWS
answered 3 months ago
  • Hello Eric. It's not working. As I said, I deleted all endpoints, configurations and models. I had only those inference component objects.

    An error occurred (ValidationException) when calling the DeleteInferenceComponent operation: Could not find endpoint "DEMO-1706784905-e5b4-endpoint".

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions