Redshift connection timeout

0

Trying to load data from dynamicFrame to redshift(using Notebook ETL), but Im getting an SQLException: Py4JJavaError: An error occurred while calling o282.getJDBCSink. : java.sql.SQLException: Amazon Error setting/closing connection: Connection timed out.

I have already configured vpc for glue to access redshift

df_final = df_converted.toDF()

from awsglue.dynamicframe import DynamicFrame

df_f=DynamicFrame.fromDF(df_final, glueContext, "test_nest")
glueContext.write_dynamic_frame.from_jdbc_conf(
    frame = df_f, 
    catalog_connection = "redshift_connection",
    connection_options = rds_conn_options,
    redshift_tmp_dir = "s3://test-glue-temp/rds/")
kls
已提問 10 個月前檢視次數 3219 次
2 個答案
1

If you have already configured VPC for glue to access it should not through timeout error, But timeout could suggest that security group on redshift is not configure to accept the request.

Here are a few troubleshooting steps you can try:

  1. Check the security group rules to allow inbound/outbound traffic between Glue and Redshift.
  2. You can try connecting to the Redshift cluster using a SQL client or running a simple query to verify its availability by launching an instance in same subnet you are using for glue.
  3. Increase connection timeout: You can try increasing the connection timeout value in your Glue job's connection options. Add the connectTimeout parameter to the rds_conn_options dictionary with a higher value, such as connectTimeout=120000 (milliseconds).

Example:

rds_conn_options = {
    "database": "your_database",
    "dbtable": "your_table",
    "user": "your_username",
    "password": "your_password",
    "url": "your_redshift_url",
    "connectTimeout": 120000
}
已回答 10 個月前
1

Hey Nitin

Thank you for the guidance. As suggested I did try reading from the redshift, I dont have any issues with it and Im able to fetch the data. Im still facing the same connectivity issue(increased the timeout period) while writing the data. Attaching the vpc screenshot for reference:

Inbound rule

Outbound rule

kls
已回答 10 個月前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南