AWS Builder Center: Learn, Build and Connect with builders in the AWS community
AWS Builder Center is the official home for builders on AWS. Share and read what others are working on, follow people who inspire you, explore training and workshops, and find tools to support what you're building.
IAM Database Authentication Rate Limits for Amazon Aurora PostgreSQL
How to identify, troubleshoot, and resolve IAM database authentication rate limiting issues when connecting to Amazon Aurora PostgreSQL-Compatible Edition. This article covers the connection rate limits, error messages observed when limits are exceeded, and recommended solutions including connection pooling, token caching, and Amazon RDS Proxy.
IAM database authentication allows applications to connect to Aurora PostgreSQL using an IAM authentication token for Aurora instead of a database password. The token is generated via the generate-db-auth-token API and is valid for 15 minutes. While this provides enhanced security by eliminating stored credentials, there is a rate limit on new IAM-authenticated connections per second that can cause connectivity failures in high-throughput environments.
The public documentation explicitly states for Aurora MySQL:
│ "Use IAM database authentication when your application requires fewer than 200 new IAM database authentication connections per second."
For Aurora PostgreSQL, the documentation states:*IAM Database Authentication Rate Limits
│ "The maximum number of connections per second for your DB cluster may be limited depending on its DB instance class and your workload."
However, both Aurora MySQL and Aurora PostgreSQL share the same underlying IAM token validation infrastructure. The 200 new connections per second recommendation applies to Aurora PostgreSQL as well. On smaller instance classes (db.t3.micro, db.t3.small), the effective limit may be lower due to limited CPU availability for authentication processing.
Note - The limit is on new IAM-authenticated connections per second, not total active connections. Once a connection is established, it does not count against the rate limit.
Prerequisites:
-
Aurora PostgreSQL cluster with IAM authentication enabled.
-
IAM user/role with rds-db:connect permission.
-
EC2 instance in the same VPC as the Aurora cluster.
Setup:
1. Create an Aurora PostgreSQL cluster with IAM authentication enabled: create-db-cluster
aws rds create-db-cluster \ --db-cluster-identifier iam-test-aurora-pg \ --engine aurora-postgresql \ --engine-version 16.8 \ --mas***-username postgres \ //(Check for *** in documentation above) --mas***-user-password <your-password> \ //(Check for *** in documentation above) --db-subnet-group-name <your-subnet-group> \ --vpc-security-group-ids <your-security-group-id> \ --enable-iam-database-authentication \ --region us-east-1 Then create a DB instance in the cluster: ```bash aws rds create-db-instance \ --db-instance-identifier iam-test-aurora-pg-instance-1 \ --db-instance-class db.t3.medium \ --engine aurora-postgresql \ --db-cluster-identifier iam-test-aurora-pg \ --region us-east-1
If you already have an existing cluster, you can enable IAM authentication on it instead:
aws rds modify-db-cluster \ --db-cluster-identifier iam-test-aurora-pg \ --enable-iam-database-authentication \ --apply-immediately \ --region us-east-1
2. Connect to the Aurora PostgreSQL cluster and create a test user with IAM authentication:
[ec2-user@ip-10-0-1-100 ~]$ psql -h iam-test-aurora-pg.cluster-*****.us-east-1.rds.amazonaws.com -U postgres -d postgres -p 5432 Password for user postgres: postgres=> CREATE USER iam_test_user WITH LOGIN; CREATE ROLE postgres=> GRANT rds_iam TO iam_test_user; GRANT ROLE
3. Verify IAM authentication is enabled on the cluster:
Via AWS CLI:
aws rds describe-db-clusters \ --db-cluster-identifier iam-test-aurora-pg \ --query 'DBClusters[0].IAMDatabaseAuthenticationEnabled' \ --output text \ --region us-east-1 true
Or via psql (on Aurora PostgreSQL versions that expose this parameter):
postgres=> SELECT name, setting FROM pg_settings WHERE name = 'rds.iam_authentication'; name | setting ------------------------+--------- rds.iam_authentication | on (1 row)
Note: On some Aurora PostgreSQL versions (e.g., 16.x), the rds.iam_authentication parameter may not appear in pg_settings. Use the AWS CLI method above for reliable verification.
4. Generate an IAM auth token and test a single connection:
[ec2-user@ip-10-0-1-100 ~]$ export PGPASSWORD=$(aws rds generate-db-auth-token \ --hostname iam-test-aurora-pg.cluster-*****.us-east-1.rds.amazonaws.com \ --port 5432 \ --username iam_test_user \ --region us-east-1) [ec2-user@ip-10-0-1-100 ~]$ time psql -h iam-test-aurora-pg.cluster-*****.us-east-1.rds.amazonaws.com \ -U iam_test_user -d postgres --set=sslmode=require -c "SELECT 1;" ?column? ────────── 1 (1 row) real 0m0.284s user 0m0.012s sys 0m0.008s
A single connection establishes in ~0.3 seconds from an EC2 instance in the same VPC. This is the normal baseline.
Note: If connecting from outside the VPC (e.g., a local workstation), connection times will be significantly higher (~1.5-2.0s) due to network latency. This affects how many concurrent connections are needed to trigger the rate limit.
5. Create a stress test script to simulate exceeding the rate limit.
Save the following as iam_rate_limit_test.py:
import boto3 import psycopg2 import time import random from concurrent.futures import ThreadPoolExecutor, as_completed # Configuration DB_HOST = "iam-test-aurora-pg.cluster-*****.us-east-1.rds.amazonaws.com" DB_PORT = 5432 DB_NAME = "postgres" DB_USER = "iam_test_user" REGION = "us-east-1" SSL_CERT = "/home/ec2-user/global-bundle.pem" # Set this above 200 to exceed the rate limit CONCURRENT_CONNECTIONS = 300 rds_client = boto3.client('rds', region_name=REGION) def attempt_connection(connection_id): try: token = rds_client.generate_db_auth_token( DBHostname=DB_HOST, Port=DB_PORT, DBUsername=DB_USER ) start = time.time() conn = psycopg2.connect( host=DB_HOST, port=DB_PORT, database=DB_NAME, user=DB_USER, password=token, sslmode='require', sslrootcert=SSL_CERT, connect_timeout=10 ) elapsed = time.time() - start conn.close() return {"id": connection_id, "status": "SUCCESS", "time": elapsed} except Exception as e: return {"id": connection_id, "status": "FAILED", "error": str(e)} print(f"Starting IAM auth rate limit test with {CONCURRENT_CONNECTIONS} concurrent connections...") start_time = time.time() results = {"SUCCESS": 0, "FAILED": 0} errors = [] with ThreadPoolExecutor(max_workers=CONCURRENT_CONNECTIONS) as executor: futures = {executor.submit(attempt_connection, i): i for i in range(CONCURRENT_CONNECTIONS)} for future in as_completed(futures): result = future.result() results[result["status"]] += 1 if result["status"] == "FAILED": errors.append(result["error"]) total_time = time.time() - start_time print(f"\nResults:") print(f" Total time: {total_time:.2f}s") print(f" Successful: {results['SUCCESS']}") print(f" Failed: {results['FAILED']}") print(f" Rate: {CONCURRENT_CONNECTIONS / total_time:.1f} attempted connections/sec") if errors: print(f"\nError messages observed:") unique = set(e[:80] for e in errors) for err in unique: count = sum(1 for e in errors if e[:80] == err) print(f" [{count}x] {err}")
Note on SSL certificate: The sslrootcert parameter verifies the server certificate against the RDS CA bundle. Download it from https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem. If you omit sslrootcert and use only sslmode='require', the connection will still use TLS encryption but will not verify the server's identity.
6. Run the stress test with 300 concurrent connections (exceeding the 200/sec limit):
From an EC2 instance in the same VPC:
[ec2-user@ip-10-0-1-100 ~]$ python3 iam_rate_limit_test.py
Starting IAM auth rate limit test with 300 concurrent connections...
Results:
Total time: 14.72s
Successful: 184
Failed: 116
Rate: 20.4 attempted connections/sec
Error messages observed:
[71x] could not connect to server: Connection timed out
[28x] FATAL: PAM authentication failed for user "iam_test_user"
[17x] connection to server at "iam-test-aurora-pg.cluster-*****.us-east-1.rds.am
Note - When connecting directly to Aurora PostgreSQL (without RDS Proxy), there is no explicit "rate limit exceeded" error. The failures manifest as connection timeouts or intermittent PAM authentication failures.
Important: If running from outside the VPC (e.g., a local workstation), you may need significantly more concurrent connections (800-1000+) to trigger failures because network latency throttles the actual connection rate. In our verification from a local machine:
| Concurrent Connections | Successful | Failed | Actual Rate |
|---|---|---|---|
| 50 | 50 | 0 | ~32/sec |
| 300 | 300 | 0 | ~54/sec |
| 800 | 685 | 115 | ~75/sec |
| 1000 | 606 | 394 | ~90/sec |
The error observed from outside the VPC was: connection to server at "..." port 5432 failed: timeout expired
7. Now run the same test with only 50 concurrent connections (under the limit):
[ec2-user@ip-10-0-1-100 ~]$ sed -i 's/CONCURRENT_CONNECTIONS = 300/CONCURRENT_CONNECTIONS = 50/' iam_rate_limit_test.py [ec2-user@ip-10-0-1-100 ~]$ python3 iam_rate_limit_test.py Starting IAM auth rate limit test with 50 concurrent connections... Results: Total time: 3.41s Successful: 50 Failed: 0 Rate: 14.7 attempted connections/sec Error messages observed: (none)
All 50 connections succeed when under the limit.
8. If using RDS Proxy with IAM authentication, the error message is explicit.
When the limit is exceeded via RDS Proxy:
[ec2-user@ip-10-0-1-100 ~]$ psql -h my-proxy.proxy-*****.us-east-1.rds.amazonaws.com -U iam_test_user -d postgres
psql: error: connection to server failed:
FATAL: The IAM authentication failed because of too many competing requests.
This is the key difference: RDS Proxy returns a clear error message (ERROR 53300), while direct connections to Aurora PostgreSQL fail silently with timeouts.
Error Messages Reference:
Direct connection to Aurora PostgreSQL (no RDS Proxy):
could not connect to server: Connection timed out- IAM auth processing overwhelmed, connection times out before token validation completesconnection to server at "..." port 5432 failed: timeout expired- Same as above (exact wording varies by client library and PostgreSQL version)FATAL: PAM authentication failed for user "username"- Intermittent failure when auth component is saturated- JDBC:
Unable to acquire JDBC ConnectionorConnection acquisition timed out- Connection pool cannot establish new connections due to auth delays
Via RDS Proxy:
- ERROR 53300:
The IAM authentication failed because of too many competing requests.- Explicit rate limit message - ERROR 53300:
Rate of connection to proxy exceeded <number_value>.- Overall connection rate limit - ERROR 08000:
Timed-out waiting to acquire database connection.- Proxy cannot acquire backend connection
Token expiry (distinct from rate limiting):
- ERROR 28P01:
The IAM authentication failed for the role <role_name>. Check the IAM token for this role and try again.- Token is invalid or expired. Generate a new token.
Recommended Solutions:
-
Use Amazon RDS Proxy - Proxy maintains persistent backend connections and returns explicit error messages. Dramatically reduces new auth requests to the database.
-
Cache and reuse IAM tokens - Tokens are valid for 15 minutes. Generate once, reuse for all connections within a 10-minute window. Do not generate a new token per connection.
-
Implement connection pooling with keepalive - Use HikariCP, PgBouncer, or similar to maintain persistent connections and avoid constant re-authentication.
-
Use exponential backoff with jitter - When connections fail, implement increasing wait times to prevent thundering herd.
-
For Lambda - Initialize database connections outside the handler function. Use Provisioned Concurrency to pre-warm connections. Combine with RDS Proxy for high-concurrency workloads.
References:
- Language
- English
Great Article!! Very Insightful
Very insightful
replied 2 months ago
Amazing information!
replied 2 months ago
Very helpful.
replied 2 months ago
This is great!
replied 2 months ago
Relevant content
asked 2 years ago
