Help us improve the AWS re:Post Knowledge Center by sharing your feedback in a brief survey. Your input can influence how we create and update our content to better support your AWS journey.
Identify offending POD within an EKS Cluster
Hi Everyone. Does anyone know if it is possible to identify the network traffic from an offending specific POD using VPC Flow log. I have activated all VPC flow log fields such as pkt-srcaddr mentioned in VPC FLOW fields but the VPC log always shows the Node IP, not the IP address from the specific POD.
THE EKS ARCHITECTURE:
NODE IP: 192.168.47.183
I have the 3 pods below each one with its ip address.
I'm generating different connection from inside of each one.. For example:
From inside of SHELL-DEMO(192.168.47.183) i have executed a curl http2demo.io(95.168.192.200) AND curl pudim.com.br(85.31.231.84) and from inside of
SHELL-DEMO3(192.168.54.139) i did a curl pudim.com.br(85.31.231.84)
I was expecting the flow logs showing the connections from the POD's ip... but when i search the VPC Flow logs the pkt-srcaddr always shows my NODE IP and not the IP from each pod that was responsible for the connection.
- Newest
- Most votes
- Most comments
Hi,
Let's quickly review how networking typically works in a Kubernetes environment, especially with Amazon EKS:
- Pod IP visibility: In a standard Kubernetes networking model, pod IPs are typically only visible within the cluster. The Node IP is used for external communication.
- VPC Flow Logs limitations: VPC Flow Logs capture information about IP traffic going to and from network interfaces in your VPC. However, they operate at the EC2 instance (node) level, not at the pod level.
- Network Address Translation (NAT): When pods communicate with external resources, their traffic is usually NAT'ed through the node's IP address. This is why you're seeing the node IP (192.168.47.183) in the VPC Flow Logs instead of individual pod IPs.
Given these constraints, it's generally not possible to directly identify specific pod traffic using VPC Flow Logs alone. The logs will show traffic from the node IP, as you've observed.
Please consider alternative approaches like enabling AWS VPC CNI Plugin with Pod ENI (Secondary ENI Allocation), Kubernetes Network Policies, Service Mesh and Container level logging and so on.
Hope it helps.
Relevant content
- asked 3 years ago
- AWS OFFICIALUpdated 6 months ago
