AWS ECS Task Definition Stuck on DELETE_IN_PROGRESS

Question

I had an ECS cluster, service, task definitions, ECR repo, RDS database, security groups, and other EC2 services, and I managed to delete all except for a single task definition and the associated network and VPC services. I can't delete any network and other VPC services before the task definition is deleted. However, the task definition is stuck on DELETE_IN_PROGRESS.

I deleted the stacks in CloudFormation but it didn't help. CloudTrails don't show any errors either. The task definition had a task role and task execution role but both are deleted. It had one container but the ECR repo had been deleted as well. I tried using AWS CLI and some --force commands to delete other associated services but none could be deleted before this task gets deleted.

* When I try to delete the VPC I get `VPC contains one or more in-use network interfaces.`
* When I try to detach the network interface with or without force I get `Failed to detach the network interface. API error: "You do not have permission to access the specified resource."` even though I am with the root user, it somehow tries to detach it with the IAM. 
* When I try to delete it, I get `The network interface can't be deleted. Network interface is currently in use.`

Here is a list of services I checked:

Running services:

EC2 Security Groups - 1
    EC2 VPCs - 1
    Subnets - 2
    Route Tables - 1
    Security Groups - 1
    Network ACLs - 1

No running services:

EC2 Instances
    EC2 Key Pairs
    EC2 Elastic IPs
    EC2 Instances
    RDS Instances
    ECS Clusters
    ECS Task Definitions
    ECR Repositories
    Lambda Functions
    S3 Buckets

Basically, I want to delete all services. Any suggestions on how to do it?

myronix88 · Answer

Do you have admin rights?

containerised · Answer

There is a possibility that a ECS service is force deleted and when you delete a service, if there are still running tasks that require cleanup, the service status moves from ACTIVE to DRAINING, and the service is no longer visible in the console, only after all tasks frmo this service have transitioned to either STOPPING or STOPPED status, the service status then moves from DRAINING to INACTIVE and operation is  completed.

You can still use `DescribeServices` API operation to find status of a service which is in the `DRAINING` or `INACTIVE` status.

While any of deployment or task is utilising deleted task definition, they will remain in `DELETE_IN_PROGRESS` state.

Please check out [ECS Deleting a task definition revision documentation](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/delete-task-definition-v2.html#resource-block-delete) for possible reasons and remediation steps.

AWS ECS Task Definition Stuck on DELETE_IN_PROGRESS

관련 콘텐츠