By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

How do I troubleshoot namespaces in a terminated state in my Amazon EKS cluster?

2 minute read
1

I tried to delete a namespace in my Amazon Elastic Kubernetes Service (Amazon EKS) cluster. However, the namespace is stuck in the "Terminating" status.

Short description

To delete a namespace, Kubernetes must first delete all the resources in the namespace. Then, it must check registered API services for the status. A namespace gets stuck in Terminating status for the following reasons:

  • The namespace contains resources that Kubernetes can't delete.
  • An API service has a False status.

Resolution

1.    Save a JSON file like in the following example:

kubectl get namespace TERMINATING_NAMESPACE -o json > tempfile.json

Note: Replace TERMINATING_NAMESPACE with the name of your stuck namespace.

2.    Remove the finalizers array block from the spec section of the JSON file:

"spec": {
        "finalizers": [
            "kubernetes"
        ]
    }

After you remove the finalizers array block, the spec section of the JSON file looks like this:

"spec" : {
    }

3.    To apply the changes, run the following command:

kubectl replace --raw "/api/v1/namespaces/TERMINATING_NAMESPACE/finalize" -f ./tempfile.json

Note: Replace TERMINATING_NAMESPACE with the name of your stuck namespace.

4.    Verify that the terminating namespace is removed:

kubectl get namespaces

Repeat these steps for any remaining namespaces that are stuck in the Terminating status.

Related information

What is Amazon EKS?

Namespaces (on the Kubernetes site)

Topics
Containers
Tags
Amazon Elastic Kubernetes Service
Language
English
AWS OFFICIAL
AWS OFFICIALUpdated 2 years ago
4 Comments

Your friendly one-liner. Make sure you don't terminate the wrong thing. I put in a safety check to make sure the variable is at least not empty. Just set TERMINATING_NAMESPACE at the beginning (I had a few to delete, hence the automation).

TERMINATING_NAMESPACE=bad-namespace-here && echo "Force Terminating: ${TERMINATING_NAMESPACE}..." && echo "${TERMINATING_NAMESPACE}" != "" && kubectl get namespace ${TERMINATING_NAMESPACE} -o json | jq 'del(.spec.finalizers)' | kubectl replace --raw "/api/v1/namespaces/${TERMINATING_NAMESPACE}/finalize" -f -
Rob
replied a year ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

profile pictureAWS
MODERATOR
AWS Official
replied a year ago

Sometimes even after going through the steps (1-4) mentioned in the repost article, the namespace remains stuck in terminating status. However, I was able to use the one-liner command provided in the comment by Rob to delete the same namespace that was stuck in terminating status and can confirm that it works.

As far as I can tell, the one-liner command does the same thing as the steps provided in the article (which is remove the finalizers from the object) but just in one line. The only requirement would be to install jq, if you don't have it installed already (yum install jq).

Tapan
replied 5 months ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

profile pictureAWS
EXPERT
AWS Official
replied 5 months ago

Relevant content