Redshift connection timeout

0

Trying to load data from dynamicFrame to redshift(using Notebook ETL), but Im getting an SQLException: Py4JJavaError: An error occurred while calling o282.getJDBCSink. : java.sql.SQLException: Amazon Error setting/closing connection: Connection timed out.

I have already configured vpc for glue to access redshift

df_final = df_converted.toDF()

from awsglue.dynamicframe import DynamicFrame

df_f=DynamicFrame.fromDF(df_final, glueContext, "test_nest")
glueContext.write_dynamic_frame.from_jdbc_conf(
    frame = df_f, 
    catalog_connection = "redshift_connection",
    connection_options = rds_conn_options,
    redshift_tmp_dir = "s3://test-glue-temp/rds/")
kls
已提问 10 个月前3231 查看次数
2 回答
1

If you have already configured VPC for glue to access it should not through timeout error, But timeout could suggest that security group on redshift is not configure to accept the request.

Here are a few troubleshooting steps you can try:

  1. Check the security group rules to allow inbound/outbound traffic between Glue and Redshift.
  2. You can try connecting to the Redshift cluster using a SQL client or running a simple query to verify its availability by launching an instance in same subnet you are using for glue.
  3. Increase connection timeout: You can try increasing the connection timeout value in your Glue job's connection options. Add the connectTimeout parameter to the rds_conn_options dictionary with a higher value, such as connectTimeout=120000 (milliseconds).

Example:

rds_conn_options = {
    "database": "your_database",
    "dbtable": "your_table",
    "user": "your_username",
    "password": "your_password",
    "url": "your_redshift_url",
    "connectTimeout": 120000
}
已回答 10 个月前
1

Hey Nitin

Thank you for the guidance. As suggested I did try reading from the redshift, I dont have any issues with it and Im able to fetch the data. Im still facing the same connectivity issue(increased the timeout period) while writing the data. Attaching the vpc screenshot for reference:

Inbound rule

Outbound rule

kls
已回答 10 个月前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则