Redshift connection timeout

0

Trying to load data from dynamicFrame to redshift(using Notebook ETL), but Im getting an SQLException: Py4JJavaError: An error occurred while calling o282.getJDBCSink. : java.sql.SQLException: Amazon Error setting/closing connection: Connection timed out.

I have already configured vpc for glue to access redshift

df_final = df_converted.toDF()

from awsglue.dynamicframe import DynamicFrame

df_f=DynamicFrame.fromDF(df_final, glueContext, "test_nest")
glueContext.write_dynamic_frame.from_jdbc_conf(
    frame = df_f, 
    catalog_connection = "redshift_connection",
    connection_options = rds_conn_options,
    redshift_tmp_dir = "s3://test-glue-temp/rds/")
kls
asked 9 months ago3002 views
2 Answers
1

If you have already configured VPC for glue to access it should not through timeout error, But timeout could suggest that security group on redshift is not configure to accept the request.

Here are a few troubleshooting steps you can try:

  1. Check the security group rules to allow inbound/outbound traffic between Glue and Redshift.
  2. You can try connecting to the Redshift cluster using a SQL client or running a simple query to verify its availability by launching an instance in same subnet you are using for glue.
  3. Increase connection timeout: You can try increasing the connection timeout value in your Glue job's connection options. Add the connectTimeout parameter to the rds_conn_options dictionary with a higher value, such as connectTimeout=120000 (milliseconds).

Example:

rds_conn_options = {
    "database": "your_database",
    "dbtable": "your_table",
    "user": "your_username",
    "password": "your_password",
    "url": "your_redshift_url",
    "connectTimeout": 120000
}
answered 9 months ago
1

Hey Nitin

Thank you for the guidance. As suggested I did try reading from the redshift, I dont have any issues with it and Im able to fetch the data. Im still facing the same connectivity issue(increased the timeout period) while writing the data. Attaching the vpc screenshot for reference:

Inbound rule

Outbound rule

kls
answered 9 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions