Issue to connect to Postgres RDS using AWS Glue

0

Hello, I'm trying to set up a Lake Formation to keep some data from RDS. Using AWS documentation and tutorial I created:

  1. AWS user with administrative permissions, AmazonAthenaFullAccess, AWSGlueConsoleFullAccess, and AWSLakeFormationDataAdmin.
  2. Lake formation
  3. AWS Glue connection to RDS using JDBC driver, VPC and RDS security group After that I tried to Test that connection, use it in Blueprint, and Crawler. Unfortunately, every time, after several minutes of work, it is Fail. The error is : Getting error: [1308dfbe-8bf9-494c-aa47-b57b07ec3713] ERROR : Internal Service Exception. Any ideas how to fix that issue?
2개 답변
0

Hello Andrey,

error description is not that specific. You could try some ways to resolve issue such as-

  • Check if RDS security group allows ingress traffic on the database port (e.g 3306) from the Glue VPC security group. Glue needs to connect to RDS from there.

  • Verify the RDS endpoint, database name, username, and password are all correct in the Glue connection. Test the connection parameters directly from a client like SQL Workbench.

  • Recheck networking/routing between the Glue VPC, RDS subnet groups. Ensure there is a VPC peering connection or routes set up.

  • increase the timeout for the Glue connection test. The default can sometimes be too short with networking delays. Try say 3 -4 mins.

  • Check CloudWatch Logs for specific log failures from Glue during the connection test. Details could help point to root causes.

  • Try creating a simpler Glue table/database pointing to RDS to test first. this will rule out other moving parts such as crawlers etc.

  • Check for service limits in Glue and Rds

Hope this helps, pls let me know if works out. Thanks

AWS
전문가
Ben Lee
답변함 7달 전
  • Hi Ben, Thank you for your reply.

    1. My RDS security group allows All Traffic inbound and outbound
    2. Everything RDS endpoint, database name, username, and password are all correct in the Glue connection are correct. No issues to connect to it from EC2 or Postgres DB Gui tools
    3. Glue connection uses the same VPC and subnet group as RDS
    4. This is not a timeout issue because connection test ran, at least 3 min before Fail
    5. The message, ERROR : Internal Service Exception, is from CloudWatch
    6. I created a new RDS with one table( 4 rows) and tried to connect it to the Glue. Result is the same
    7. How can I check for service limits? Thanks again for your reply
0

Ensure the glue connection is on a private subnet with either a route to a NAT gateway or you have setup the glue vpc endpoint.

This may resolve your issue.

profile picture
전문가
답변함 7달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인