How to reduce AWS NLB target group registration time for UDP pods on EKS

Question

Hello,

I'm trying to reduce the target group registration time of an AWS Network Load Balancer (NLB) for pods running UDP-based applications on Amazon EKS.

To speed up target registration when pods are created or restarted, I’ve already tuned the target group’s health check settings as follows:

Healthy threshold: 2 consecutive successes
Unhealthy threshold: 2 consecutive failures
Timeout: 2 seconds
Interval: 5 seconds

Based on these settings, I would expect new targets to become healthy in roughly 10 seconds (2 × 5s interval).
However, in practice, I consistently observe a 40–50 second delay before the target is registered and starts receiving traffic.

Questions:
Is this longer registration time expected behavior for NLB target groups?
Are there any additional settings or best practices to reduce the registration time?

Any insights or tuning tips would be greatly appreciated. Thanks!
Kyoungho

re:Post Agent · Answer

For Network Load Balancers (NLBs) with UDP-based applications on EKS, the target registration time you're experiencing is longer than what your health check settings would suggest.

There are a few important points to understand:

1. For newly registered targets, only a single successful health check is required to consider the target healthy, regardless of the healthy threshold count setting. The healthy threshold count only applies when a target is transitioning from an unhealthy state back to a healthy state.

2. The delay you're experiencing might be related to other factors beyond just the health check settings. When a new pod comes up in your EKS cluster, there are several processes that need to complete before it's fully registered with the NLB target group.

To reduce the registration time, consider these approaches:

1. Verify that your pods are actually ready to receive traffic when they start up. If your application takes time to initialize, the health checks will fail until initialization is complete.

2. You could modify the target group attributes using the AWS CLI. For example, you might want to adjust the deregistration delay timeout:

```
aws elbv2 modify-target-group-attributes --target-group-arn  --attributes Key=deregistration_delay.timeout_seconds,Value=
```

3. Consider implementing lifecycle hooks in your Auto Scaling group if you're using one, which can help manage the transition states of your instances.

4. For EKS specifically, ensure that your readiness probes are properly configured and aligned with your NLB health check expectations.

5. Check if there are any network policies or security groups that might be delaying the health check communication.

The discrepancy between your expected 10-second registration time and the observed 40-50 seconds suggests there might be additional factors in your EKS environment affecting the registration process. It could be worth investigating if there are any EKS-specific settings or container startup procedures contributing to this delay.  
**Sources**  
[Optimize load balancer health check parameters for Amazon ECS - Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/load-balancer-healthcheck.html)  
[Use ModifyTargetGroupAttributes with a CLI - AWS SDK Code Examples](https://docs.aws.amazon.com/code-library/latest/ug/elastic-load-balancing-v2_example_elastic-load-balancing-v2_ModifyTargetGroupAttributes_section.html)  
[Target Group - Only send traffic to healthy nodes | AWS re:Post](https://repost.aws/questions/QU_qxkkmJiQ_mMJClbgOLaRw/target-group-only-send-traffic-to-healthy-nodes)

How to reduce AWS NLB target group registration time for UDP pods on EKS

Add your answer

Relevant content