En utilisant AWS re:Post, vous acceptez les AWS re:Post Conditions d’utilisation
AWS re:Postre:Post

get_connection timeout in AWG Glue job

0

I am following the articale below for doing update in MySQL using pymysql - https://awstip.com/aws-etl-glue-job-insert-update-support-7a396db832b. However, it looks like the job is timing out on below line -

connection = glue_client.get_connection(Name="<My Connection>")

I do not see any exception in the logs.

Test connection works fine.Also, the same connection worked when I used it in another job for insert-only created from Visual editor.

Sujets
Analytique
Balises
AWS Glue
Langue
English
rePost-User-0629761
demandé il y a 2 ans217 vues
1 réponse
1

To identify the exact error messages, searching through the ERROR logs with GlueExceptionAnalysisListener typically would lead me to the exact line that errors and the Error message. Please try that and let us know if you are still unable to identify the error

profile pictureAWS
Ananth Tirumanur
répondu il y a 2 ans
  • rePost-User-0629761
    il y a 2 ans

    Hi,

    I only see that the GlueExceptionAnalysisListener was registered in the error logs. No other exception is logged until the job times out.

    2022-10-03 21:44:07,711 INFO [Thread-12] spark.SparkContext (Logging.scala:logInfo(57)): Registered listener com.amazonaws.services.glueexceptionanalysis.GlueExceptionAnalysisListener.

    Also, note that I am using MySQL v8.

    I started fresh with a new job where the script was generated by Glue Studio. It ran fine without any issues. However, as soon as I added get_connection() call, the job started timing out.

  • Ananth Tirumanur
    il y a 2 ans

    Please refer to Glue connection page where it mentions how to use the Glue Connection https://docs.aws.amazon.com/glue/latest/dg/connection-properties.html#connection-properties-jdbc

    Note that MySQL connections are possible for Amazon RDS instances only. If you are using other MySQL database, you dont have to use Glue Connections. You would need to upload the JDBC driver to S3. In the AWS Glue job console, Choose the AWS Glue job and edit the Job details tab, under Libraries > Referenced File Path and enter the S3 URI of JDBC file.

    If you are able to get the job to work without connections and the data looks right, you do not have to add a Glue connection.

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions

Contenus pertinents