An error occurred while calling o243.pyWriteDynamicFrame. DAY_OF_MONTH

Question

Hi, we have a problem with this error.. in one job of Glue with one table. In the job be just read the RDS MySQL information of table and write in S3 bucket, we didn't make any changes and the job run successfully, but recently the job failed with this error: "An error occurred while calling o243.pyWriteDynamicFrame. DAY_OF_MONTH". We check the logs and we just see the table of this problem, We suppose than it is about of some bad format of one of the dates columns.. but we consulting some dates directly on the database and to identify the invalid value on a date column.. but we don't see anything yet

¿Any suggestion about the problem?

Answer

Hello,

It seems data related issue only. I would suggest before writing the data verify if you are getting correct data without null values and valid format. You can update your script and add following lines to verify the data.

1. Comment your write method in code
2. Use printSchema and show() method to verify the data before write method might be possible some transformation in between creating the issue(like null and incompatible)

`dyf.printSchema()`

`dyf.toDF().show(5)`

Also i would suggest to check the complete logs specially caused by to understand the exeact reason. Also using above method you will be able to find if there is any issue with data.

An error occurred while calling o243.pyWriteDynamicFrame. DAY_OF_MONTH

관련 콘텐츠