EMR Cluster Auto Termination Not Working

Hello,

As you might know that “AutoTerminationIsClusterIdle” CloudWatch metric Indicates whether the cluster qualifies for AutoTermination or not. A value of 0 indicates that the cluster is in active use by one of the following components: ——————————————————————————————————

1. An active YARN application
2. HDFS utilization greater than 10%
3. Active EMR notebook or EMR Studio connections
4. An on-cluster UI, such as the Spark History Server

For 1. You can check "AppsRunning", "AppSubmitted" or "ContainerPendingRatio" metric.

For 2. You can check "HDFSUtilization" metric.

For 3. You can check "TotalNotebookKernels" metric

There could be other reasons as well that you can check in the metric collector log in the primary node. I recommend to check all the considerations given in this document to verify them all are satisfied in your cluster. For an instance, below,

To use auto-termination, the metrics-collector process must be able to connect to the public API endpoint for auto-termination in API Gateway. If you use a private DNS name with Amazon Virtual Private Cloud, auto-termination won't function properly. To ensure that auto-termination works, we recommend that you take one of the following actions:

-Remove the API Gateway interface VPC endpoint from your Amazon VPC. -Follow the instructions in Why do I get an HTTP 403 Forbidden error when connecting to my API Gateway APIs from a VPC?[1] to disable the private DNS name setting. -Launch your cluster in a private subnet instead. For more information, see the topic on Private subnets. Please refer below docs for more details.

[+] https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-auto-termination-policy.html#emr-auto-termination-considerations

[+] https://docs.aws.amazon.com/emr/latest/ManagementGuide/UsingEMR_ViewingMetrics.html