How do I use VPC Flow Logs tag and next-hop fields to identify workloads and trace traffic paths?

Short description

VPC Flow Logs version 11 adds two field categories:

EC2 Resource Tag fields: Embed tag values from instances, network interfaces, and Auto Scaling groups directly into each flow record.

${instance-tag}       1st instance tag value
${instance-tag-2}     2nd instance tag value
${interface-tag}      1st network-interface tag value
${interface-tag-2}    2nd network-interface tag value
${asg-tag}            1st auto-scaling-group tag value
${asg-tag-2}          2nd auto-scaling-group tag value

Next-Hop Metadata fields: Show which intermediate resource (NAT Gateway, VPC Endpoint, NLB, TGW) handled each flow.

${next-hop-interface-id}     ENI ID of the next hop
${next-hop-subnet-id}        Subnet of the next hop
${next-hop-az-id}            Availability Zone of the next hop
${next-hop-vpc-id}           VPC of the next hop
${next-hop-interface-type}   nat_gateway | network_load_balancer | regional_nat_gateway | transit_gateway | vpc_endpoint

---

Resolution

Step 1: Create a flow log with the new fields

The tag fields require a TagFieldSpecifications parameter that maps each tag field position to a specific tag key. The following example uses boto3, you can also create the flow log from the VPC console.

import boto3

ec2 = boto3.client('ec2', region_name='us-east-1')

response = ec2.create_flow_logs(
    ResourceIds=['vpc-0a1b2c3d4e5f67890'],
    ResourceType='VPC',
    TrafficType='ALL',
    LogDestinationType='cloud-watch-logs',
    LogGroupName='/vpc/flow-logs',
    DeliverLogsPermissionArn='arn:aws:iam::ACCOUNT_ID:role/VPCFlowLogsRole',
    LogFormat='${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status} ${instance-tag} ${instance-tag-2} ${interface-tag} ${next-hop-interface-id} ${next-hop-interface-type} ${next-hop-az-id}',
    TagFieldSpecifications=[
        {
            'ResourceType': 'instance',
            'TagKeys': ['Name', 'Environment']
        },
        {
            'ResourceType': 'network-interface',
            'TagKeys': ['Name']
        }
    ]
)

Note: Tag fields require ec2:DescribeTags and iam:CreateServiceLinkedRole permissions. A Service Linked Role and EventBridge Managed Rules are created automatically.

Step 2: Read the flow log records

Once the flow log is active and traffic is flowing, records appear in CloudWatch Logs. The following examples show how the new fields look for different traffic types:

# Egress to internet via NAT Gateway:
11 338797193989 eni-0cd6c73986ed7a21c 10.50.2.131 8.8.8.8 0 0 1 3 252 1784235370 1784235383 ACCEPT OK flowlogs%2Dtest%2Dinstance production flowlogs%2Dtest%2Deni eni-0071b16f98b585221 nat_gateway use1-az6

# Egress to VPC Endpoint:
11 338797193989 eni-0cd6c73986ed7a21c 10.50.2.131 10.50.2.102 56160 443 6 82 103814 1784235370 1784235383 ACCEPT OK flowlogs%2Dtest%2Dinstance production flowlogs%2Dtest%2Deni eni-03c36206818484c86 vpc_endpoint use1-az6

# Ingress from VPC Endpoint — next-hop shows the sending ENI:
11 338797193989 eni-0cd6c73986ed7a21c 10.50.2.102 10.50.2.131 443 56160 6 80 4160 1784235370 1784235383 ACCEPT OK flowlogs%2Dtest%2Dinstance production flowlogs%2Dtest%2Deni eni-03c36206818484c86 vpc_endpoint use1-az6

# Ingress from internet — next-hop is '-' (IGW has no ENI):
11 338797193989 eni-0cd6c73986ed7a21c 8.8.8.8 10.50.2.131 0 0 1 3 252 1784235370 1784235383 ACCEPT OK flowlogs%2Dtest%2Dinstance production flowlogs%2Dtest%2Deni - - -

Field order: version account-id interface-id srcaddr dstaddr srcport dstport protocol packets bytes start end action log-status instance-tag instance-tag-2 interface-tag next-hop-interface-id next-hop-interface-type next-hop-az-id

Step 3: Interpret the next-hop behavior

Traffic type	next-hop-interface-type	Meaning
Egress to internet	nat_gateway	Traffic exits through NAT GW
Egress to VPC Endpoint	vpc_endpoint	Traffic goes to SSM/S3/etc endpoint
Ingress from VPC Endpoint	vpc_endpoint	Shows the ENI that sent the traffic
Ingress from internet	-	IGW has no ENI — field not available

Important:

• Tag values are UTF-8 percent-encoded (hyphens become %2D: flowlogs-test-instance → flowlogs%2Dtest%2Dinstance).
• Tags may show - during the first hour after subscription creation.
• Next-hop shows - when the last hop has no ENI (IGW) or for ingress from some services (TGW, NLB).

---

Limitations

EC2 Resource Tag fields:

• Maximum 2 tag keys per resource type.
• Tags are not computed for resources owned by a different account (shared subnets).
• A Service Linked Role and EventBridge Managed Rules are auto-created, do not delete them manually.
• ASG tags require at least one enabled CloudTrail trail.
• Tags may be missing during the first hour after creation.

Next-Hop fields:

• Not available when the next hop has no ENI (e.g., IGW).
• Not available for cross-region traffic.
• Not available for ingress from some services (TGW, NLB).
• Not computed if the next-hop ENI is owned by a different account (except next-hop-az-id).
• For middle-boxes, shows the middle-box ENI, not the final destination.

---

Conclusion

With EC2 resource tag support, you can embed tag values from your network interfaces, EC2 instances, and auto scaling groups. This eliminates the need for you to join flow log data with separate tag metadata to correlate records with specific workloads. With next-hop metadata support, you can capture details about the next-hop network interface for each flow, including its interface ID, subnet, Availability Zone, VPC, and interface type. These fields help you understand how traffic traverses through network resources such as NAT Gateways, Network Load Balancers, and Transit Gateways without requiring manual correlation of multiple data sources.

---

Related information

[1] Amazon VPC Flow Logs introduces additional metadata (What's New)

[2] VPC Flow Log records — Available fields

[3] Publishing flow logs to CloudWatch Logs

[4] VPC Flow Logs limitations

[5] How do I set up VPC flow logs?

[6] How do I troubleshoot VPC flow logs?