Allow only CloudFlare IPs on EC2 behind an elastic load balancer

0

Hi,

as per title, I need to allow only CloudFlare IPs on an EC2 instance that is behind an elastic load balancer. Here's my current setup:

I have a domain (mydomain.com) through which I expose different services via third-level subdomains (eg. service1.mydomain.com). CloudFlare manages this domain, redirecting every request to my ELB, which, in turn, forwards requests to the appropriate target group based on the subdomain.

For instance, my ELB has a rule that redirects every request to service1.mydomain.com to the target group service1-tg which, for simplicity, contains only one EC2 instance (eg. Service1EC2Instance).

This EC2 instance has a public IP, and I want to prevent direct access via its public IP address.

To achieve this, I've created two security groups:

  • AllowOnlyFromCloudFlareIPs: this sg contains inbound rules that allows only CloudFlare's IPs. I've used prefix lists with IPs obtained from https://www.cloudflare.com/en-gb/ips/.
  • AllowOnlyFromLoadBalancer: this sg contains a single inbound rule with the ELB's security group as source.

My initial attempt was to associate the sg AllowOnlyFromLoadBalancer with the Service1EC2Instance EC2 instance. I expected to lose access via the public IP but still have access via my domain name (service1.mydomain.com) because all traffic would pass through CloudFlare first and then through the ELB. However, with the AllowOnlyFromLoadBalancer sg only my EC2 instance became entirely unreachable and I cannot understand why.

In my second attempt, I also assocated the sg AllowOnlyFromCloudFlareIPs to the Service1EC2Instance EC2 instance, and to my surprise, the service become reachable via service1.mydomain.com while remaining unreachable via its public IP address.

That's exactly the behavior I want, but I'm curious about why it works this way. Why is it necessary to allow CloudFlare's IP addresses on my EC2 instance and not on my ELB?

Edit: I forgot to mention that the EC2 instance requires both AllowOnlyFromCloudFlareIPs and AllowOnlyFromLoadBalancer security groups to work as I want. If either of them is missing, the instance becomes entirely unreachable.

Gigitsu
asked 7 months ago1050 views
6 Answers
1

Hello, from what you write, I understand that:

  • With the AllowOnlyFromLoadBalancer security group applied to your EC2, it becomes unreachable from CloudFlare IPs.
  • With the AllowOnlyFromCloudFlareIPs security group applied to your EC2, it becomes reachable from CloudFlare IPs.

Are you sure that, from CloudFlare, you are pointing to the ALB and not directly to the public IP address of your EC2 instance?

AWS
Vincent
answered 7 months ago
  • I was just about to write a much longer response - but this is what I think as well.

0

Hello @Vincent, thank you.

Are you sure that, from CloudFlare, you are pointing to the ALB and not directly to the public IP address of your EC2 instance?

Yes, I'm sure that CloudFlare is pointing to the ALB and not directly to the public IP.

One thing I did not mention in my question is that the EC2 instance requires both security groups to work as I want. If either of them is missing, the instance becomes entirely unreachable.

I will edit my original question to add this information too.

Gigitsu
answered 7 months ago
0

Is it an application load balancer or a network load balancer? If it's an NLB then these preserve the source IPs, so from the EC2's perspective the traffic will look like it's all coming from CloudFlare IPs.

This would be consistent with the behaviour you describe.

https://aws.amazon.com/elasticloadbalancing/network-load-balancer/

Preserve source IP address

Network Load Balancer preserves the client side source IP allowing the back-end to see the IP address of the client. This can then be used by applications for further processing.

profile picture
EXPERT
Steve_M
answered 7 months ago
0

Thank you @Steve, it's an application load balancer as far as I can tell from the console.

Enter image description here

Gigitsu
answered 7 months ago
0

As its an ALB that means it has to be web traffic, so there will be a webserver listening on the EC2. Do the webserver logs offer any clue as to the source of the incoming traffic, I mean are they seen as coming from the ALB's private IP or the CloudFlare IPs?

Also, what rules are in the security group that's associated with the ALB? I'm guessing it will be something like inbound 80 & 443 from CloudFlare IPs, and then outbound 80 to the EC2's private IP? Or does the outbound rule reference one or other of the SGs AllowOnlyFromCloudFlareIPs or AllowOnlyFromLoadBalancer? Whichever of these SGs is in the ALB's rule is going to have to be associated with the EC2 in order for traffic to be allowed (and I think it should be AllowOnlyFromLoadBalancer).

If the ALB outbound rule references one of those SGs, how about replacing that rule (just for troubleshooting purposes) with the private IP of the EC2 and see if it improves things?

profile picture
EXPERT
Steve_M
answered 7 months ago
0

As its an ALB that means it has to be web traffic, so there will be a webserver listening on the EC2.

Exactly, there is an apache web server.

Do the webserver logs offer any clue as to the source of the incoming traffic

That's a good idea. I edited httpd config to have the following log format: LogFormat "'Forwarded for - %{X-Forwarded-For}i' %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined And I got the following log: 'Forwarded for - my.ip.address, cf.ip.address' alb.private.ip.address - - [21/Sep/2023:12:25:51 +0000] ...

So it seems that incoming traffic is coming from ALB's private IP while, in order, my laptop IP address and a CF ip address are listed in the forwarded for section.

Also, what rules are in the security group that's associated with the ALB?

There are 2 inbound rules:

  • type: HTTP source: 0.0.0.0/0 (any ip)
  • type: All Traffic source: the alb security group itself

Ant there is 1 outbound rule:

  • type: All Traffic destination: 0.0.0.0/0

There is no cloud flare rule associated to the ALB.

If the ALB outbound rule references one of those SGs, how about replacing that rule (just for troubleshooting purposes) with the private IP of the EC2 and see if it improves things?

I can try to add an outbound rule with the privete IP of the EC2 and associate it with the ALB. But this is risky because we have a bunch of services behind this ALB and for some of them I cannot have downtime.

Gigitsu
answered 7 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions