Glue ETL job to load data from S3 to Redshift failed

0

I am running a Glue ETL job to load data from parquet file from S3 into Redshift table. Enter image description here

I am able to get data preview while configuring connections but job fails with below error.

Error Category: UNCLASSIFIED_ERROR; Failed Line Number: 31; An error occurred while calling o112.pyWriteDynamicFrame. Exception thrown in awaitResult:

Please let me issue and provide solution.

Ani
gefragt vor 4 Monaten311 Aufrufe
2 Antworten
1
Akzeptierte Antwort

I would recommend you switch to the script and edit it to add a few things:

  1. print out the schema of the dynamic frame of the source. See if that matches your Redshift schema.
  2. Look for nested fields/structures - you might need a Resolve choice
  3. use an Apply mapping transform if your schema doesn't match Redshift
  4. check permissions of the IAM role of your job
beantwortet vor 4 Monaten
profile picture
EXPERTE
überprüft vor 4 Monaten
  • Thank you. Issue was in schema difference between Source and Redshift.

0

The data preview won't really insert into Redshift, which is what is really failing, likely a permission issue.
Look in the log for a cause, also looking into the Redshift error table might help (if it got that far): https://docs.aws.amazon.com/redshift/latest/dg/r_STL_LOAD_ERRORS.html

profile pictureAWS
EXPERTE
beantwortet vor 4 Monaten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen