AWS Glue test connection failing with error message "Failed to test connection gtn-rds-postgres-conn due to FAILED status"


I have created a database instance in RDS postgres having below details in the screenshot Enter image description here

Security group inbound rules Enter image description here

Security group outbound rules Enter image description here

Endpoint Enter image description here

Connection details Enter image description here

Connection test Enter image description here

AWSGlueServiceRole_S3_RDS_policies Enter image description here

Error message Enter image description here

First 2 lines of Log message from CloudWatch 2023-06-27T12:10:36.435+05:30 ERROR StatusLogger Unrecognized format specifier [d]

2023-06-27T12:10:36.439+05:30 ERROR StatusLogger Unrecognized conversion specifier [d] starting at position 16 in conversion pattern.

Looking for help on this issue.

asked a year ago2023 views
4 Answers

What version of postgress are you using? Glue seems to use a postgresql driver incompatible with Postgresql14 by default because of scram as mentioned

You can follow a thread here how to resolve this issue. One option is to downgrade to v13.x

Also, you are aware that your RDS is public and accessible over the internet along with the Glue connection connecting to the Public IP Address of your RDS Instance with your configuration as it stands.

profile picture
answered a year ago

The tester is pending a driver upgrade and it's not able to handle scram authentication, which RDS will use by default.
Although is strange that your error log doesn't give you a clear message like: "org.postgresql.util.PSQLException: The authentication type 10 is not supported."

In practice, the tester is a convenience, the jobs and crawlers use their own drivers, so I suggest you give it a try to see if the issue is just with the tester.

profile pictureAWS
answered a year ago
  • Tried to run crawler but its failing with below error log 2023-06-27T13:50:58.524+05:30 [28ce2a66-4766-4e1b-8176-b2e3bec8dcd3] BENCHMARK : Running Start Crawl for Crawler rds-postgres-gtn-crawler 2023-06-27T13:51:45.894+05:30 [28ce2a66-4766-4e1b-8176-b2e3bec8dcd3] ERROR : Test connection failed for connection 'gtn-rds-postgres-conn'


Having the same issue. Trying to get data from DynamoDB to an Aurora V2 Postgres instance. If I use the "Test Connection" feature on the added connection I get "The authentication type 10 is not supported", then if I try to run the crawler anyway I get "Test connection failed for connection ...". So it appears as though the test is running even for the crawler. It looks like Postgres 14 came out about 2 years ago. Maybe I'm missing something, but it seems strange for one of AWS' flagship products to be 2 years behind in compatibility with one of the world's most popular database engines. Does moving from scram back to md5 pose a security risk? Maybe this is why AWS isn't interested in changing it? Because changing the configuration is a valid workaround? I assumed the change to scram was about improving security, but maybe that assumption is wrong.

answered 8 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions