Redshift connection timeout

0

Trying to load data from dynamicFrame to redshift(using Notebook ETL), but Im getting an SQLException: Py4JJavaError: An error occurred while calling o282.getJDBCSink. : java.sql.SQLException: Amazon Error setting/closing connection: Connection timed out.

I have already configured vpc for glue to access redshift

df_final = df_converted.toDF()

from awsglue.dynamicframe import DynamicFrame

df_f=DynamicFrame.fromDF(df_final, glueContext, "test_nest")
glueContext.write_dynamic_frame.from_jdbc_conf(
    frame = df_f, 
    catalog_connection = "redshift_connection",
    connection_options = rds_conn_options,
    redshift_tmp_dir = "s3://test-glue-temp/rds/")
kls
질문됨 10달 전3217회 조회
2개 답변
1

If you have already configured VPC for glue to access it should not through timeout error, But timeout could suggest that security group on redshift is not configure to accept the request.

Here are a few troubleshooting steps you can try:

  1. Check the security group rules to allow inbound/outbound traffic between Glue and Redshift.
  2. You can try connecting to the Redshift cluster using a SQL client or running a simple query to verify its availability by launching an instance in same subnet you are using for glue.
  3. Increase connection timeout: You can try increasing the connection timeout value in your Glue job's connection options. Add the connectTimeout parameter to the rds_conn_options dictionary with a higher value, such as connectTimeout=120000 (milliseconds).

Example:

rds_conn_options = {
    "database": "your_database",
    "dbtable": "your_table",
    "user": "your_username",
    "password": "your_password",
    "url": "your_redshift_url",
    "connectTimeout": 120000
}
답변함 10달 전
1

Hey Nitin

Thank you for the guidance. As suggested I did try reading from the redshift, I dont have any issues with it and Im able to fetch the data. Im still facing the same connectivity issue(increased the timeout period) while writing the data. Attaching the vpc screenshot for reference:

Inbound rule

Outbound rule

kls
답변함 10달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠