Skip to content

IAM Database Authentication Rate Limits for Amazon Aurora PostgreSQL

8 minute read
Content level: Advanced
6

How to identify, troubleshoot, and resolve IAM database authentication rate limiting issues when connecting to Amazon Aurora PostgreSQL-Compatible Edition. This article covers the connection rate limits, error messages observed when limits are exceeded, and recommended solutions including connection pooling, token caching, and Amazon RDS Proxy.

IAM database authentication allows applications to connect to Aurora PostgreSQL using an IAM authentication token for Aurora instead of a database password. The token is generated via the generate-db-auth-token API and is valid for 15 minutes. While this provides enhanced security by eliminating stored credentials, there is a rate limit on new IAM-authenticated connections per second that can cause connectivity failures in high-throughput environments.

The public documentation explicitly states for Aurora MySQL:

│ "Use IAM database authentication when your application requires fewer than 200 new IAM database authentication connections per second."

For Aurora PostgreSQL, the documentation states:*IAM Database Authentication Rate Limits

│ "The maximum number of connections per second for your DB cluster may be limited depending on its DB instance class and your workload."

However, both Aurora MySQL and Aurora PostgreSQL share the same underlying IAM token validation infrastructure. The 200 new connections per second recommendation applies to Aurora PostgreSQL as well. On smaller instance classes (db.t3.micro, db.t3.small), the effective limit may be lower due to limited CPU availability for authentication processing.

Note - The limit is on new IAM-authenticated connections per second, not total active connections. Once a connection is established, it does not count against the rate limit.

Prerequisites:

  1. Aurora PostgreSQL cluster with IAM authentication enabled.

  2. IAM user/role with rds-db:connect permission.

  3. EC2 instance in the same VPC as the Aurora cluster.

Setup:

1. Create an Aurora PostgreSQL cluster with IAM authentication enabled: create-db-cluster

aws rds create-db-cluster \
 --db-cluster-identifier iam-test-aurora-pg \
 --engine aurora-postgresql \
 --engine-version 16.8 \
 --mas***-username postgres \        //(Check for *** in documentation above)
 --mas***-user-password <your-password> \   //(Check for *** in documentation above)
 --db-subnet-group-name <your-subnet-group> \
 --vpc-security-group-ids <your-security-group-id> \
 --enable-iam-database-authentication \
 --region us-east-1

Then create a DB instance in the cluster:
 
```bash
aws rds create-db-instance \
 --db-instance-identifier iam-test-aurora-pg-instance-1 \
 --db-instance-class db.t3.medium \
 --engine aurora-postgresql \
 --db-cluster-identifier iam-test-aurora-pg \
 --region us-east-1

If you already have an existing cluster, you can enable IAM authentication on it instead:

aws rds modify-db-cluster \
 --db-cluster-identifier iam-test-aurora-pg \
 --enable-iam-database-authentication \
 --apply-immediately \
 --region us-east-1

2. Connect to the Aurora PostgreSQL cluster and create a test user with IAM authentication:

[ec2-user@ip-10-0-1-100 ~]$ psql -h iam-test-aurora-pg.cluster-*****.us-east-1.rds.amazonaws.com -U postgres -d postgres -p 5432
 
Password for user postgres:
 
postgres=> CREATE USER iam_test_user WITH LOGIN;
CREATE ROLE
 
postgres=> GRANT rds_iam TO iam_test_user;
GRANT ROLE

3. Verify IAM authentication is enabled on the cluster:

Via AWS CLI:

aws rds describe-db-clusters \
 --db-cluster-identifier iam-test-aurora-pg \
 --query 'DBClusters[0].IAMDatabaseAuthenticationEnabled' \
 --output text \
 --region us-east-1
 
true

Or via psql (on Aurora PostgreSQL versions that expose this parameter):

postgres=> SELECT name, setting FROM pg_settings WHERE name = 'rds.iam_authentication';
         name         | setting
------------------------+---------
 rds.iam_authentication | on
(1 row)

Note: On some Aurora PostgreSQL versions (e.g., 16.x), the rds.iam_authentication parameter may not appear in pg_settings. Use the AWS CLI method above for reliable verification.

4. Generate an IAM auth token and test a single connection:

[ec2-user@ip-10-0-1-100 ~]$ export PGPASSWORD=$(aws rds generate-db-auth-token \
 --hostname iam-test-aurora-pg.cluster-*****.us-east-1.rds.amazonaws.com \
 --port 5432 \
 --username iam_test_user \
 --region us-east-1)
 
[ec2-user@ip-10-0-1-100 ~]$ time psql -h iam-test-aurora-pg.cluster-*****.us-east-1.rds.amazonaws.com \
 -U iam_test_user -d postgres --set=sslmode=require -c "SELECT 1;"
 
 ?column?
──────────
       1
(1 row)
 
real   0m0.284s
user   0m0.012s
sys    0m0.008s

A single connection establishes in ~0.3 seconds from an EC2 instance in the same VPC. This is the normal baseline.

Note: If connecting from outside the VPC (e.g., a local workstation), connection times will be significantly higher (~1.5-2.0s) due to network latency. This affects how many concurrent connections are needed to trigger the rate limit.

5. Create a stress test script to simulate exceeding the rate limit.

Save the following as iam_rate_limit_test.py:

import boto3
import psycopg2
import time
import random
from concurrent.futures import ThreadPoolExecutor, as_completed
 
# Configuration
DB_HOST = "iam-test-aurora-pg.cluster-*****.us-east-1.rds.amazonaws.com"
DB_PORT = 5432
DB_NAME = "postgres"
DB_USER = "iam_test_user"
REGION = "us-east-1"
SSL_CERT = "/home/ec2-user/global-bundle.pem"
 
# Set this above 200 to exceed the rate limit
CONCURRENT_CONNECTIONS = 300
 
rds_client = boto3.client('rds', region_name=REGION)
 
def attempt_connection(connection_id):
   try:
       token = rds_client.generate_db_auth_token(
           DBHostname=DB_HOST, Port=DB_PORT, DBUsername=DB_USER
       )
       start = time.time()
       conn = psycopg2.connect(
           host=DB_HOST, port=DB_PORT, database=DB_NAME,
           user=DB_USER, password=token,
           sslmode='require', sslrootcert=SSL_CERT,
           connect_timeout=10
       )
       elapsed = time.time() - start
       conn.close()
       return {"id": connection_id, "status": "SUCCESS", "time": elapsed}
   except Exception as e:
       return {"id": connection_id, "status": "FAILED", "error": str(e)}
 
print(f"Starting IAM auth rate limit test with {CONCURRENT_CONNECTIONS} concurrent connections...")
start_time = time.time()
 
results = {"SUCCESS": 0, "FAILED": 0}
errors = []
 
with ThreadPoolExecutor(max_workers=CONCURRENT_CONNECTIONS) as executor:
   futures = {executor.submit(attempt_connection, i): i for i in range(CONCURRENT_CONNECTIONS)}
   for future in as_completed(futures):
       result = future.result()
       results[result["status"]] += 1
       if result["status"] == "FAILED":
           errors.append(result["error"])
 
total_time = time.time() - start_time
 
print(f"\nResults:")
print(f" Total time: {total_time:.2f}s")
print(f" Successful: {results['SUCCESS']}")
print(f" Failed:    {results['FAILED']}")
print(f" Rate:      {CONCURRENT_CONNECTIONS / total_time:.1f} attempted connections/sec")
 
if errors:
   print(f"\nError messages observed:")
   unique = set(e[:80] for e in errors)
   for err in unique:
       count = sum(1 for e in errors if e[:80] == err)
       print(f" [{count}x] {err}")

Note on SSL certificate: The sslrootcert parameter verifies the server certificate against the RDS CA bundle. Download it from https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem. If you omit sslrootcert and use only sslmode='require', the connection will still use TLS encryption but will not verify the server's identity.

6. Run the stress test with 300 concurrent connections (exceeding the 200/sec limit):

From an EC2 instance in the same VPC:

[ec2-user@ip-10-0-1-100 ~]$ python3 iam_rate_limit_test.py
 
Starting IAM auth rate limit test with 300 concurrent connections...
 
Results:
 Total time: 14.72s
 Successful: 184
 Failed:    116
 Rate:      20.4 attempted connections/sec
 
Error messages observed:
 [71x] could not connect to server: Connection timed out
 [28x] FATAL: PAM authentication failed for user "iam_test_user"
 [17x] connection to server at "iam-test-aurora-pg.cluster-*****.us-east-1.rds.am

Note - When connecting directly to Aurora PostgreSQL (without RDS Proxy), there is no explicit "rate limit exceeded" error. The failures manifest as connection timeouts or intermittent PAM authentication failures.

Important: If running from outside the VPC (e.g., a local workstation), you may need significantly more concurrent connections (800-1000+) to trigger failures because network latency throttles the actual connection rate. In our verification from a local machine:

Concurrent ConnectionsSuccessfulFailedActual Rate
50500~32/sec
3003000~54/sec
800685115~75/sec
1000606394~90/sec

The error observed from outside the VPC was: connection to server at "..." port 5432 failed: timeout expired

7. Now run the same test with only 50 concurrent connections (under the limit):

[ec2-user@ip-10-0-1-100 ~]$ sed -i 's/CONCURRENT_CONNECTIONS = 300/CONCURRENT_CONNECTIONS = 50/' iam_rate_limit_test.py
 
[ec2-user@ip-10-0-1-100 ~]$ python3 iam_rate_limit_test.py
 
Starting IAM auth rate limit test with 50 concurrent connections...
 
Results:
 Total time: 3.41s
 Successful: 50
 Failed:    0
 Rate:      14.7 attempted connections/sec
 
Error messages observed:
 (none)

All 50 connections succeed when under the limit.

8. If using RDS Proxy with IAM authentication, the error message is explicit.

When the limit is exceeded via RDS Proxy:

[ec2-user@ip-10-0-1-100 ~]$ psql -h my-proxy.proxy-*****.us-east-1.rds.amazonaws.com -U iam_test_user -d postgres
 
psql: error: connection to server failed:
 FATAL: The IAM authentication failed because of too many competing requests.

This is the key difference: RDS Proxy returns a clear error message (ERROR 53300), while direct connections to Aurora PostgreSQL fail silently with timeouts.

Error Messages Reference:

Direct connection to Aurora PostgreSQL (no RDS Proxy):

  • could not connect to server: Connection timed out - IAM auth processing overwhelmed, connection times out before token validation completes
  • connection to server at "..." port 5432 failed: timeout expired - Same as above (exact wording varies by client library and PostgreSQL version)
  • FATAL: PAM authentication failed for user "username" - Intermittent failure when auth component is saturated
  • JDBC: Unable to acquire JDBC Connection or Connection acquisition timed out - Connection pool cannot establish new connections due to auth delays

Via RDS Proxy:

  • ERROR 53300: The IAM authentication failed because of too many competing requests. - Explicit rate limit message
  • ERROR 53300: Rate of connection to proxy exceeded <number_value>. - Overall connection rate limit
  • ERROR 08000: Timed-out waiting to acquire database connection. - Proxy cannot acquire backend connection

Token expiry (distinct from rate limiting):

  • ERROR 28P01: The IAM authentication failed for the role <role_name>. Check the IAM token for this role and try again. - Token is invalid or expired. Generate a new token.

Recommended Solutions:

  1. Use Amazon RDS Proxy - Proxy maintains persistent backend connections and returns explicit error messages. Dramatically reduces new auth requests to the database.

  2. Cache and reuse IAM tokens - Tokens are valid for 15 minutes. Generate once, reuse for all connections within a 10-minute window. Do not generate a new token per connection.

  3. Implement connection pooling with keepalive - Use HikariCP, PgBouncer, or similar to maintain persistent connections and avoid constant re-authentication.

  4. Use exponential backoff with jitter - When connections fail, implement increasing wait times to prevent thundering herd.

  5. For Lambda - Initialize database connections outside the handler function. Use Provisioned Concurrency to pre-warm connections. Combine with RDS Proxy for high-concurrency workloads.

References:

5 Comments

Great Article!! Very Insightful

AWS
SUPPORT ENGINEER

replied 2 months ago

Very insightful

AWS

replied 2 months ago

Amazing information!

AWS

replied 2 months ago

Very helpful.

replied 2 months ago

This is great!

replied 2 months ago