Glue connection to Secure RDS Postgres instance

Question

The main goal is to use AWS Glue to get data from S3 and push it to a Postgres instance on RDS.

The main problem I have is not being able to connect to the RDS instance
I have ssh tunneling to RDS where within the same VPC there are 2 subnets and Security Groups one has the RDS instance and the other has EC2 instance the RDS SG allows only postgres connections originating from the EC2 SG and nowhere else, the EC2 SG allows only SSH connections.

Now in this case when I try using glue to create a normal connection with JDBC the connection is created but fails to connect to RDS instance, I have tried the EC2 SG and Subnet and creating a new SG with correct permissions but still no luck.

Let me know what else I can check and please ask any questions that can help you identify the issue.

re:Post Agent · Answer

Based on your setup, there are several potential issues preventing AWS Glue from connecting to your RDS PostgreSQL instance.

The main challenge appears to be your network security configuration. Since your RDS security group only allows PostgreSQL connections from the EC2 security group and nowhere else, Glue cannot directly connect to your RDS instance. Here are some key areas to check:

1. **Security Group Configuration**: AWS Glue needs its own security group that allows it to connect to the RDS instance. You'll need to modify your RDS security group to allow inbound connections on port 5432 from the Glue security group, not just from the EC2 security group.

2. **Network Connectivity**: Ensure that AWS Glue is configured to run in the same VPC as your RDS instance. Glue needs to be in a subnet that can route traffic to the RDS subnet.

3. **Bidirectional Connectivity**: Apache Spark (which Glue uses) requires bidirectional connectivity. Your Glue security group should have self-referencing inbound rules to allow connectivity between Glue workers.

4. **IAM Permissions**: Verify that the IAM role used by Glue has the necessary permissions to access both the Glue service and the RDS instance.

5. **JDBC URL Format**: Confirm your JDBC URL is correctly formatted for PostgreSQL.

6. **Database Credentials**: Ensure the username and password in your Glue connection are correct.

Since you're using SSH tunneling for other connections, note that AWS Glue doesn't natively support SSH tunneling to connect to databases. You'll need to modify your security architecture to allow direct connections from Glue to RDS.

Consider using the AWSSupport-TroubleshootGlueConnection runbook in AWS Systems Manager to help identify specific connection issues between Glue and your RDS instance.  
**Sources**  
[Troublehsoot AWS Glue connection to a JDBC data source | AWS re:Post](https://repost.aws/knowledge-center/glue-troubleshoot-connection-jdbc-source)  
[AWS Glue Connection to RDS SQL Server Fails – "Failed status due to failed connection" | AWS re:Post](https://repost.aws/questions/QUxFTw3i6NSJyRAyKtD1G8fQ/aws-glue-connection-to-rds-sql-server-fails-failed-status-due-to-failed-connection)  
[Troubleshooting connections to your RDS for PostgreSQL instance - Amazon Relational Database Service](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ConnectToPostgreSQLInstance.Troubleshooting.html)  
[Troubleshooting connection issues in AWS Glue - AWS Glue](https://docs.aws.amazon.com/glue/latest/dg/troubleshooting-connection.html)

Glue connection to Secure RDS Postgres instance

Add your answer

Relevant content