Observing Pod to Pod network flows in EKS Cluster

Question

I need to find network flows between pods, both pods from different nodes and pods from same node as well. I tried VPC Flow logs, but it only provides pod IPs and no identifier of the pod (e.g. pod id, labels).  Another limitation VPC Flow logs doesn't provide flows within the same node. Also, I tried Container Insight as well, but it does not give flow information. Is there any way we can find network flows with src and destination pod information? Is there a way I can use Amazon VPC CNI for getting this information? I do not wish to use Cilium or Calico or any eBPF-based agent.

AWS-User-7455027 · Answer

AWS Detective allows you to examine VPC flow log information for your EKS workloads, display visual summaries about these network flows, and aggregate information by EKS pods.

https://aws.amazon.com/about-aws/whats-new/2023/01/amazon-detective-vpc-flow-logs-visualizations-eks-workloads/

Mstein-AWS · Answer

Hi!

To start with, check out [this article](**https://aws.amazon.com/blogs/networking-and-content-delivery/using-vpc-flow-logs-to-capture-and-query-eks-network-communications/**) for a great solution to address your challenge with a little extension.  Very simply, the EKS cluster is deployed with worker nodes (EC2) and these have one or more ENIs attached to accommodate the pods deployed with the VPC CNI.

You will always need to map the IP with the pod name in the VPC which you can see using this command:

```
kubectl get pods --all-namespaces -o wide
```
The main issue is that these can change if the pod migrates, gets restarted etc.  You could use the solution above and extend it further with Athena to merge the flowlogs data with a "pod name to IP mapping" document to create the picture you are looking for.    See the [INSERT INTO](https://docs.aws.amazon.com/athena/latest/ug/insert-into.html) function for some ideas.

Observing Pod to Pod network flows in EKS Cluster

Relevant content