Create an EKS Cluster without using NATGW but using VPC EKS Endpoint

0

In order to avoid the cost of the NATGW Data transfer charges, I wanted to create an EKS Cluster without creating the NAT GW. I tried this, but the Private Nodes failed to join the cluster

  • Created the VPC with Private & Public Subnet

  • Created the Internet GW

  • Created the VPC EKS Endpoint interface

  • Tried to create the EKS Cluster using the eksctl utility, but creation failed due to Private Nodes unable to join the cluster. Even though I have the VPC EKS Endpoints, not sure why it failed to join the cluster.

  • Also one more question, how do you specify the VPC EKS Endpoint in the yaml file, so that connection from Private subnet Node goes through VPC EKS Endpoint rather than through NatGW

asked 6 months ago772 views
2 Answers
0

To make private endpoints work instead of public APi then ensure private dns is enabled when setting up the endpoints. This changes the api IP address returned by the route 53 resolver. You don’t need to change any configurations for any services. It’s all controlled by DNS.

The other option is to run all your EC2’s in a public subnet and enable public IPs. All API services will be available via the internet without a NAT gateway.

It’s likely you will need a few private endpoints and not just EKS to which point it may be cheaper to go NAT gateway or put all the hosts in public subnet.

profile picture
EXPERT
answered 6 months ago
  • Private DNS is enabled for VPC EKS Endpoint as shown below. Still I am seeing the failure as Private Nodes are not able to reach API Server.

    Private DNS names enabled Yes

    Private DNS names eks.us-east-2.amazonaws.com

    Also, If I add the NatGW, Cluster creation is successful, But I don't want to add the NATGW, as I really wanted without NATGW

  • VPC endpoints are needed for all the AWS services that your nodes and Pods need to communicate with. Examples include Amazon ECR, Elastic Load Balancing, Amazon CloudWatch, AWS Security Token Service, and Amazon Simple Storage Service (Amazon S3). The endpoint must include the subnet that the nodes are in. Not all AWS services support VPC endpoints.

  • Yes I understand that, But within a VPC, if you have multiple Subnets (Private & Public), it is not required (and cannot) to create VPC endpoints on every private subnet. Because within the VPC local traffic is default allowed. (Meaning if your VPC is 10.100.0.0/16, your private subnets cidrs are in AZ1 are viz.. 10.100.20.0/24, 10.100.24.0/24, 10.100.28.0/24, & 10.100.32.0/24, then we can only create VPC EKS Interface endpoints in only one subnet in that AZ1. And by default entire VPC Range (10.10.0.0/16) will go through the VPC Endpoint only. (Please correct my understanding, if any)

    I have created VPC EKS Endpoints on one private subnet in each AZ (AZ1 & AZ2). Even after this, the Cluster creation is failing where private nodes are not able to reach EKS API Server.

    As per my knowledge, we can create VPC endpoints (Either interface/Gateway depending on the Service) for most of the services. Like Cloudwatch, ECR, ECS, EKS and others. Once I create them, I do not need any explicit changes in my VPC routing as it is at the subnet and VPC level, I hope my understanding is correct.

0

You need your EKS cluster endpoint to be private so that eks nodes in your private subnet can access the EKS API server to join the kubernetes cluster. The EKS VPC endpoint is slightly different.

Amazon EKS supports making calls to all of its API actions through the interface endpoint, but not to the Kubernetes APIs. The Kubernetes API server already supports a private endpoint.

The challenge you might face is that you might want to administer the EKS cluster via kubectl from outside the VPC, at which point, you might want to enable the EKS cluster endpoint to be both public and private - https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html

AWS
Oli
answered 6 months ago
  • Sorry , it is all coming back to same question. I am not sure, what I am missing from your answer. If I use VPC Wizard and create private/public subnets without NatGW and then I am using the eksctl utility with clustet-config.yaml file mentioning vpc/subnets/ami family and other details. I have also enabled privateNetworking=true which means that node will be created in a private subnet with no Public IP addresses for the Nodes. If I try to create this eks cluster, what other settings I need to make for the worker nodes not to fail while trying to join the cluster.

    I believe, whatever the private/public cluster endpoint which you are referring is for kubectl to connect to API Server within the VPC/or outside the VPC.[I am fine with only connecting to API Server via kubectl from within the VPC, over the internal network]

    Definitely need help in understanding how to create a EKS Cluster without NatGW when privateNetworking is enabled in the eksctl yaml file.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions