Troubleshoot NLB UDP flows

Question

Hello,

I want to setup a facing internet NLB to loadbalance UDP flows to onprem servers (doesn't sound like a good idea, but it's just for a migration period).

Here is the full flow:

client -> NLB -> VPN (Opnsense wireguard) -> server on prem

I've created a target group and target for my onprem server (private ip), using UPD port for traffic and TCP port for healthcheck. The target is marked as healthy, meaning NLB -> server on prem TCP conneciton works well. For now the UDP flow doesn't, I don't receive anything on the onprem server when I send UDP packet to the NLB listener.

On the NLB monitoring tab, I can see some active flow count UDP, telling me that the NLB is receiving my attemps. Client / Target reset counts are 0, but it seems to have many Loadbalancer reset count which I'm not sure how to interpret (might be part of the issue).

Finally, in order to make sure all the network / routes, VPN configuration was ok, I started a fresh EC2 box in the same public subnet my NLB is setup. From this, I tried to send UDP packets to my onprem server directly using the same port configured in the target group / target (acting as the NLB somehow) and it worked well.

So I'm not sure where I should look at next, does anyone have an idea ?

Thanks a lot !

Accepted Answer

According to this blog https://aws.amazon.com/blogs/aws/new-udp-load-balancing-for-network-load-balancer/ NLB does not support IP target types with UDP.
Supported Targets – UDP on Network Load Balancers is supported for Instance target types (IP target types and PrivateLink are not currently supported).

Answer

Actually the Network Load Balancer supports IP-based Target group listening to UDP port .
The blog can be outdated since 2019, so I suggest to refer to AWS documentation here: https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-target-groups.html#target-type

![Enter image description here](/media/postImages/original/IM6n2cy257TgCRw2lCU-LvUA)

I did a quick test in my account using netcat on ec2 instances. I could successfully send UDP packets using a remote machine to a public NLB and have the UDP packet routed to the instance in an IP Target group type.
- You can test hosting your listeners in your VPC first. Ensure the listeners are on both AZs where the NLB eni's are. 
- You can also test **enabling cross-AZ load balancing** for your NLB since you are only concerned with UDP traffic.
- If you don't have cross-AZ Load balancing enabled, you can start testing UDP traffic to NLB eni where you have a host listening in the same AZ.
- Enable VPC flow logs to investigate how the traffic transfers and whether rejected at any point.

These considerations apply to your use-case:
- If you register a target by IP address and the IP address is in the same VPC as the load balancer, the load balancer verifies that it is from a subnet that it can reach.
- For UDP and TCP_UDP target groups, do not register instances by IP address if they reside outside of the load balancer VPC or if they use one of the following instance types: C1, CC1, CC2, CG1, CG2, CR1, G1, G2, HI1, HS1, M1, M2, M3, or T1. Targets that reside outside the load balancer VPC or use an unsupported instance type might be able to receive traffic from the load balancer but then be unable to respond.
- When using on premises resources, the IP addresses of these targets must still come from one of the following CIDR blocks:
    - 10.0.0.0/8 (RFC 1918)
    - 100.64.0.0/10 (RFC 6598)
    - 172.16.0.0/12 (RFC 1918)
    - 192.168.0.0/16 (RFC 1918)

If you find this helpful, please accept the answer.

Troubleshoot NLB UDP flows

Relevant content