Mounting Amazon S3 Files on EC2 Instances in Private Subnets Without Internet Access

Prerequisites

• An AWS account with an Amazon Simple Storage Service (Amazon S3) general purpose bucket with versioning turned on
• An S3 file system created and linked to the bucket
• An Amazon Virtual Private Cloud (Amazon VPC) with a private subnet that has no route to an internet gateway
• An Amazon EC2 instance running Amazon Linux 2023 in the private subnet
• The amazon-efs-utils package version 3.0.0 or above installed on the Amazon EC2 instance
• The botocore Python package installed on the Amazon EC2 instance
• An IAM instance profile with the AmazonS3FilesClientFullAccess managed policy attached

Key takeaways

• On a private Amazon EC2 instance, skip the S3 Gateway endpoint and mount with the nodirects3read option. This is the simplest working configuration: one fewer VPC endpoint to manage, no 5-second per-mount stall, and reads delivered at expected NFS throughput.
• The S3 Gateway endpoint is still a valid alternative if you need the fastest path for large sequential reads (files 1 MiB or larger) and you want the Amazon EC2 instance role to be the caller on S3 access logs.
• Do not combine "no Gateway endpoint" with default mount options. Every fresh mount pays a 5-second stall while the direct-read path times out. Either add the endpoint or add the mount flag.
• Files 128 KiB or smaller are served from high-performance storage on first directory access. The Gateway endpoint has no effect on them.
• Writes are always handled by the S3 Files service role through AWS-internal infrastructure that does not traverse the customer VPC.

Overview of steps

1. Create the required VPC endpoints
2. Create the mount target in the private subnet
3. Configure the Amazon EC2 instance IAM role
4. Install dependencies and mount
5. Test file operations

Step 1: Create the required VPC endpoints

The S3 Files mount helper communicates with several AWS services during the mount process. In a private subnet without internet access, you must create VPC endpoints for each service.

Important: The Amazon S3 Gateway endpoint is not required. Mount with the nodirects3read option instead. Without this option, the mount helper's local proxy tries a direct Amazon S3 GET on every fresh mount, waits 5 seconds for the request to fail, then falls back to the service-side path. Adding nodirects3read skips that direct-read path entirely.

The following table lists the required endpoints:

Endpoint	Service name	Type	Purpose
S3 Files	aws.api.REGION.s3files	Interface	S3 Files control plane API
AWS STS	com.amazonaws.REGION.sts	Interface	IAM credential retrieval for NFS authentication
Amazon CloudWatch Logs	com.amazonaws.REGION.logs	Interface	Mount helper pushes logs to the /aws/efs/utils log group
AWS Systems Manager	com.amazonaws.REGION.ssm	Interface	Systems Manager access for management
SSM Messages	com.amazonaws.REGION.ssmmessages	Interface	Session Manager connectivity
EC2 Messages	com.amazonaws.REGION.ec2messages	Interface	Run Command delivery

Replace REGION with your AWS Region code, for example us-east-1.

Create a security group for the interface endpoints that allows inbound HTTPS (TCP 443) from the Amazon EC2 security group:

VPCE_SG=$(aws ec2 create-security-group \
    --group-name "vpce-sg" \
    --description "Allow HTTPS from EC2 SG for VPC endpoints" \
    --vpc-id vpc-EXAMPLE \
    --query 'GroupId' --output text)

aws ec2 authorize-security-group-ingress \
    --group-id "${VPCE_SG}" \
    --protocol tcp --port 443 \
    --source-group "${EC2_SG}"

Create the interface endpoints. The following example uses us-east-1:

REGION="us-east-1"
VPC_ID="vpc-EXAMPLE"
SUBNET_ID="subnet-EXAMPLE"

for SERVICE in "com.amazonaws.${REGION}.sts" \
              "com.amazonaws.${REGION}.logs" \
              "com.amazonaws.${REGION}.ssm" \
              "com.amazonaws.${REGION}.ssmmessages" \
              "com.amazonaws.${REGION}.ec2messages"; do
    aws ec2 create-vpc-endpoint \
        --vpc-id "${VPC_ID}" \
        --vpc-endpoint-type Interface \
        --service-name "${SERVICE}" \
        --subnet-ids "${SUBNET_ID}" \
        --security-group-ids "${VPCE_SG}" \
        --private-dns-enabled
done

The S3 Files endpoint uses a different naming convention:

aws ec2 create-vpc-endpoint \
    --vpc-id "${VPC_ID}" \
    --vpc-endpoint-type Interface \
    --service-name "aws.api.${REGION}.s3files" \
    --subnet-ids "${SUBNET_ID}" \
    --security-group-ids "${VPCE_SG}" \
    --private-dns-enabled

Step 2: Create the mount target in the private subnet

S3 Files requires a mount target (an ENI) in the same Availability Zone as your Amazon EC2 instance. Create a security group that allows inbound NFS traffic (TCP 2049):

MT_SG=$(aws ec2 create-security-group \
    --group-name "s3files-mount-target-sg" \
    --description "Allow NFS 2049 from EC2 SG" \
    --vpc-id vpc-EXAMPLE \
    --query 'GroupId' --output text)

aws ec2 authorize-security-group-ingress \
    --group-id "${MT_SG}" \
    --protocol tcp --port 2049 \
    --source-group "${EC2_SG}"

aws s3files create-mount-target \
    --region "${REGION}" \
    --file-system-id fs-EXAMPLE \
    --subnet-id "${SUBNET_ID}" \
    --security-groups "${MT_SG}"

Step 3: Configure the Amazon EC2 instance IAM role

The instance profile needs two policies:

1. Managed policy — Attach AmazonS3FilesClientFullAccess. This provides the s3files:ClientMount, s3files:ClientWrite, and s3files:ClientRootAccess permissions.

2. Inline policy — Grant direct Amazon S3 read access for intelligent read routing:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "S3ObjectReadAccess",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:GetObjectVersion"
            ],
            "Resource": "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*"
        },
        {
            "Sid": "S3BucketListAccess",
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::DOC-EXAMPLE-BUCKET"
        }
    ]
}

Replace DOC-EXAMPLE-BUCKET with your bucket name.

Step 4: Install dependencies and mount

Connect to the instance from AWS Systems Manager Session Manager because SSH requires internet or a bastion host:

aws ssm start-session --target i-EXAMPLE --region us-east-1

Install the required packages

Because the instance has no internet access, you must use one of the following approaches to install amazon-efs-utils and botocore:

Option A: Temporary NAT gateway (simplest)

1. Create a NAT gateway in a public subnet.
2. Add a route (0.0.0.0/0 to the NAT gateway) to the private subnet's route table.
3. Install packages:

sudo yum -y install amazon-efs-utils python3-pip

sudo pip3 install botocore

4. Remove the route and delete the NAT gateway.

Option B: Download wheels and install from Amazon S3

On a machine with internet access, download the botocore package and its dependencies, then upload them to an Amazon S3 bucket:

pip3 download botocore -d /tmp/botocore-wheels/

aws s3 cp /tmp/botocore-wheels/ s3://your-bucket/botocore-wheels/ --recursive

On the private instance, download and install from Amazon S3:

aws s3 cp s3://your-bucket/botocore-wheels/ /tmp/botocore-wheels/ --recursive

sudo pip3 install --no-index --find-links /tmp/botocore-wheels/ botocore

Option C: Golden AMI (recommended for production)

Bake an AMI with amazon-efs-utils and botocore pre-installed. This eliminates runtime dependency on internet or Amazon S3 access for package installation.

Why botocore is critical

The amazon-efs-utils mount helper requires botocore to perform IAM authentication during the mount process. Without botocore, the mount helper cannot retrieve temporary credentials from AWS Security Token Service (AWS STS), and the mount fails immediately with the following error:

mount.nfs4: access denied by server while mounting 127.0.0.1:/

The botocore package is not bundled with the amazon-efs-utils RPM. You must install it separately. This is the most frequent root cause of "access denied" errors in private subnet environments.

Verify installation

python3 -c "import botocore; print(botocore.__version__)"

Expected output:

1.x.x

Mount the file system

sudo mkdir -p /mnt/s3files

sudo mount -t s3files -o nodirects3read fs-EXAMPLE:/ /mnt/s3files

The nodirects3read option is required for a private-subnet mount without an S3 Gateway endpoint. Without this option, the mount helper's local proxy tries a direct Amazon S3 GET on every fresh mount and waits 5 seconds for the request to time out before the S3 Files service takes over. Adding this flag tells the proxy to skip the direct-read path and route every read through the NFS mount target.

For persistent mounts, add the following entry to /etc/fstab:

fs-EXAMPLE:/  /mnt/s3files  s3files  _netdev,nodirects3read  0  0

Replace fs-EXAMPLE with your file system ID.

Verify the mount:

df -h /mnt/s3files

Expected output:

Filesystem      Size  Used Avail Use% Mounted on
127.0.0.1:/     8.0E     0  8.0E   0% /mnt/s3files

Step 5: Test file operations

Write a file:

echo "Hello from private subnet" | sudo tee /mnt/s3files/test.txt

Read the file back:

cat /mnt/s3files/test.txt

Write a larger file:

sudo dd if=/dev/urandom of=/mnt/s3files/large-file.bin bs=1024 count=1024

Files written to the mount point are automatically synchronized to the linked Amazon S3 bucket within approximately 30 to 60 seconds.

Minimum security configuration

The following tables define the least-privilege security group rules required for S3 Files to function in a private subnet.

Amazon EC2 instance security group (outbound only)

Direction	Protocol	Port	Destination	Purpose
Outbound	TCP	2049	Mount target security group	NFS traffic to S3 Files mount target
Outbound	TCP	443	VPC endpoint security group	HTTPS to AWS STS, Amazon CloudWatch Logs, S3 Files API endpoints

Security groups are stateful. Return traffic is automatically allowed. No inbound rules are required on the Amazon EC2 instance for S3 Files to function.

Mount target security group

Direction	Protocol	Port	Source	Purpose
Inbound	TCP	2049	Amazon EC2 instance security group	Accept NFS connections from Amazon EC2

VPC endpoint security group

Direction	Protocol	Port	Source	Purpose
Inbound	TCP	443	Amazon EC2 instance security group	Accept HTTPS from Amazon EC2 for API calls

Network ACL considerations

If you use restrictive network ACLs (NACLs), make sure the following conditions are met:

• NACLs are stateless. Both inbound and outbound directions must be explicitly allowed.
• If the Amazon EC2 instance and mount target are in the same subnet, NACLs do not filter that traffic because NACLs only apply at subnet boundaries.
• It's a best practice to place the Amazon EC2 instance and mount target in the same subnet to avoid NACL complexity for NFS traffic.
• Make sure ephemeral ports (1024-65535) are allowed inbound for return traffic from interface endpoints.
• The default NACL allows all traffic. Restrictive NACLs are the most frequent cause of connectivity issues in locked-down environments.
• No S3 prefix-list egress rule is needed with nodirects3read because the client never tries to reach Amazon S3 directly.

Troubleshooting

Mount hangs indefinitely with no error or timeout

Root cause: The mount helper tries to create an Amazon CloudWatch Logs log group (/aws/efs/utils) before establishing the NFS connection. Without a CloudWatch Logs VPC endpoint, this API call hangs indefinitely and blocks the entire mount process.

Diagnosis: Check the mount log while the mount is hanging:

sudo cat /var/log/amazon/efs/mount.log

If the log ends with the following line, the mount helper is stuck trying to reach Amazon CloudWatch Logs:

Starting new HTTPS connection (1): logs.us-east-1.amazonaws.com:443

Solution: Create the com.amazonaws.REGION.logs interface VPC endpoint with private DNS turned on.

Workaround: Turn off CloudWatch logging in the S3 Files configuration:

sudo sed -i 's/^enabled = true/enabled = false/' /etc/amazon/efs/s3files-utils.conf

access denied by server while mounting 127.0.0.1:/

This error shows that the mount helper established the TLS tunnel but NFS authentication failed. Check the following causes:

1. Missing AWS STS VPC endpoint — The mount helper uses AWS STS to obtain temporary credentials for IAM-based NFS authentication. Without the com.amazonaws.REGION.sts endpoint, credential retrieval fails.

2. Missing botocore — The mount helper requires botocore to interact with AWS services. Verify with the following command:

python3 -c "import botocore; print(botocore.__version__)"

3. Missing IAM permissions — Verify that AmazonS3FilesClientFullAccess is attached to the instance profile.

4. Mount target not in the same Availability Zone — Only one mount target per Availability Zone is allowed. Verify with the following command:

aws s3files list-mount-targets --file-system-id fs-EXAMPLE

SSM Session Manager not connecting

If AWS Systems Manager Session Manager cannot reach the instance in the private subnet, verify that all three SSM endpoints exist and are in the available state with private DNS turned on:

• com.amazonaws.REGION.ssm
• com.amazonaws.REGION.ssmmessages
• com.amazonaws.REGION.ec2messages

Run the following command to check:

aws ec2 describe-vpc-endpoints \
    --filters "Name=service-name,Values=com.amazonaws.us-east-1.ssm,com.amazonaws.us-east-1.ssmmessages,com.amazonaws.us-east-1.ec2messages" \
    --query 'VpcEndpoints[].{Service:ServiceName,State:State}' \
    --output table

Performance: nodirects3read compared to default mount options

The following table shows the measured performance difference on a 5 MB test file in a private subnet with no route to Amazon S3:

Configuration	Latency	Throughput
Default mount options (no Gateway endpoint)	5,374 ms	977 kB/s
Mount with nodirects3read (no Gateway endpoint)	244 ms	22.1 MB/s

The 5-second stall occurs because the mount helper's local proxy tries a direct Amazon S3 GET that always fails in this topology. Adding nodirects3read skips that path and delivers reads at expected NFS throughput.

If you have an S3 Gateway endpoint available and need maximum throughput for large sequential reads (files 1 MiB or larger), do not use nodirects3read. The direct-read path through the Gateway endpoint measured 26.7 MB/s on the same 5 MB file and 123 MB/s on a 50 MB file.

IAM principal on Amazon S3 access logs

The IAM principal that appears on Amazon S3 access logs depends on the mount configuration:

Configuration	IAM principal on reads
Mount with nodirects3read	S3 Files service role
Mount with S3 Gateway endpoint (default options)	Amazon EC2 instance role

Audit teams must be aware of this difference when designing service control policies (SCPs), bucket policies, or Amazon S3 access log analytics.

Conclusion

Mounting S3 Files in a private subnet requires VPC endpoints, correct security group rules, and the botocore Python package. The most frequent cause of "access denied" errors is missing botocore because the package is not bundled with amazon-efs-utils and must be installed separately. A missing Amazon CloudWatch Logs VPC endpoint causes the mount to hang indefinitely without any error because the mount helper tries to create a log group before establishing the NFS connection.

The minimum set of VPC endpoints for a successful mount is: S3 Files (aws.api.REGION.s3files), AWS STS, and Amazon CloudWatch Logs. Add the three AWS Systems Manager endpoints if you need remote management without SSH.

For a private Amazon EC2 mount, skip the S3 Gateway endpoint and mount with nodirects3read or encode the option in /etc/fstab.

Related information

• Amazon S3 Files performance
• Mounting S3 file systems
• VPC endpoints
• efs-utils and 'nodirects3read' on the GitHub website