EKS with service type NLB connection resets

Hello,

I have a P2P deployment running in EKS. I have deployed a service type of LoadBalacer to that port with the following annotations:

service.beta.kubernetes.io/aws-load-balancer-type: "external"
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"
service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"

Everything is working fine when I connect first client to the IP of the NLB but the problem happens when a second connection comes. It's getting a

read: connection reset by peer

The load balancer metrics show that the reset counters are spiking.

Connecting multiple clients work when I connect them directly to the pod or the service. Also I have checked with TCPDump that the traffic does not even flow to the pod.