Source IP visibility for combined Ingress and Egress inspection Architectures

It has been over 5 years since we launched AWS Network Firewall alongside AWS Gateway Load Balancer for third-party firewall deployments. Today, there are many AWS customers utilizing these to inspect traffic both ingressing into their VPCs from an untrusted network such as the Internet, and egressing from their VPCs to an untrusted network. Customers are choosing to deploy these inspection appliances with either a centralized ingress/egress VPC (similar to a DMZ approach on-premises) or in a distributed approach with Network Firewall or Gateway Load Balancer endpoints (GWLBe) in each VPC. You can read more about distributed or centralized approaches here, but regardless, when you choose to inspect, there can be nuances regarding visibility of the source IP in the packet if you do ingress and egress inspection in the same VPC. With traditional, on-premises firewalls, source network address translation (SNAT) and destination network address translation (DNAT) are handled on the same appliance that does the inspection, and thus the single physical endpoint can see the source IP prior to NAT occurring, regardless of direction. On AWS, inspection is decoupled from other functions such as NAT and thus for SNAT, NAT Gateway is typically responsible for that function and for DNAT, Elastic Load Balancers (ELB) are usually responsible for that function. To see the IP of the VPC elastic network interfaces that is sending the packet before it is NAT'ed to the NAT Gateway IP, we must place the GWLBe between the ENI and the NAT Gateway. To see the IP address of the internet client before it is NAT'ed from the ELB IP address towards the instance, we must place the GWLBe between the ELB and Internet Gateway (IGW).

To maintain visibility of the source IP for both ingress to a load balancer and egress through a NAT Gateway, we must deploy two firewall endpoints per availability zone. You can do this by either deploying two firewalls which can share the same policy or not, or you can use multiple firewall attachments for the same firewall. One will be used for ingress inspection and the other for egress inspection. Due to the need for separate default routes, these two endpoints will have to be deployed in different subnets. You will have at least 5 subnets per AZ: 1 for NAT Gateways, 1 for ELBs, 1 for ingress inspection endpoints, 1 for egress inspection endpoints and 1 for the private subnet where your workload resides or where the Transit Gateway (TGW) or Cloud WAN (CWAN) attachments reside if you are using this as a centralized ingress/egress VPC. The following are the route tables that will need to be configured, also shown in Figures 1&2:

• There will be an IGW route table (edge associated route table) which will have the local route in it, and route the subnet IP prefix for the ELB subnets to the ingress inspection endpoint in that availability zone. Note, you will need to route each availability zones ELB subnet to the ingress inspection GWLBe in that availability zone, so you do not create cross AZ dependencies.
• There will be a route table which can be reused on all of the ingress inspection subnets which points the default routes (0.0.0.0/0, ::/0) to the IGW and then has the local route.
• There will be a route table for each of the ELB subnets which points the default routes to the ingress GWLBe in the same AZ as the NAT Gateway to, again, prevent cross-AZ dependencies. If you are doing a centralized ingress approach, you will also need the appropriate routes to the TGW/CWAN.
• There will be a route table for each of the NAT Gateway subnets with points the default routes to the IGW and then if you are doing distributed inspection, you point the local route to the egress GWLBe in the same AZ. If you are doing centralized egress, you will need to point the routes for the TGW/CWAN to the egress GWLBe in the same AZ.
• There will be a route table for each of the egress inspection subnet which points the default routes to the NAT Gateway in the same AZ to prevent cross AZ dependency and then has the local route. If you are doing centralized egress, you will need routes for your TGW/CWAN that point to the TGW/CWAN attachment.
• There will be a route table for each of the private subnets / TGW/CWAN attachment subnets which has the local route and then points the default routes to the egress GWLBe in the same AZ to prevent cross-AZ dependencies.

Figure-1 - Centralized ingress and egress VPC

Figure-2 - Distributed ingress and egress VPC