Redshift connection timeout

0

Trying to load data from dynamicFrame to redshift(using Notebook ETL), but Im getting an SQLException: Py4JJavaError: An error occurred while calling o282.getJDBCSink. : java.sql.SQLException: Amazon Error setting/closing connection: Connection timed out.

I have already configured vpc for glue to access redshift

df_final = df_converted.toDF()

from awsglue.dynamicframe import DynamicFrame

df_f=DynamicFrame.fromDF(df_final, glueContext, "test_nest")
glueContext.write_dynamic_frame.from_jdbc_conf(
    frame = df_f, 
    catalog_connection = "redshift_connection",
    connection_options = rds_conn_options,
    redshift_tmp_dir = "s3://test-glue-temp/rds/")
kls
質問済み 10ヶ月前3214ビュー
2回答
1

If you have already configured VPC for glue to access it should not through timeout error, But timeout could suggest that security group on redshift is not configure to accept the request.

Here are a few troubleshooting steps you can try:

  1. Check the security group rules to allow inbound/outbound traffic between Glue and Redshift.
  2. You can try connecting to the Redshift cluster using a SQL client or running a simple query to verify its availability by launching an instance in same subnet you are using for glue.
  3. Increase connection timeout: You can try increasing the connection timeout value in your Glue job's connection options. Add the connectTimeout parameter to the rds_conn_options dictionary with a higher value, such as connectTimeout=120000 (milliseconds).

Example:

rds_conn_options = {
    "database": "your_database",
    "dbtable": "your_table",
    "user": "your_username",
    "password": "your_password",
    "url": "your_redshift_url",
    "connectTimeout": 120000
}
回答済み 10ヶ月前
1

Hey Nitin

Thank you for the guidance. As suggested I did try reading from the redshift, I dont have any issues with it and Im able to fetch the data. Im still facing the same connectivity issue(increased the timeout period) while writing the data. Attaching the vpc screenshot for reference:

Inbound rule

Outbound rule

kls
回答済み 10ヶ月前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ