- Newest
- Most votes
- Most comments
Hello,
The above error indicates that your Glue job running on Glue workers (version 4, G.4X, 20 workers) is attempting to use a time zone identifier "GMT000" thats not recognized by the Java runtime environment used by Spark.
There might be condition that your Python script or any libraries used in your Glue job, reference an incorrect or invalid time zone like GMT000. You can ensure for the same and replace it with a valid time zone identifier, such as GMT, UTC, or a specific time zone like America/New_York supported by Spark version 3.3.0 (used by Glue version 4). Also please check for any environment variables or configurations passed to your Glue job having incorrect time zone settings causing the issue. It is also requested to please check for the data being processed for any fields that might be incorrectly formatted with the GMT000 time zone. If you find any such fields, you can update them to use a valid time zone identifier. If you are using any third-party libraries that handle time zones, ensure they are up-to-date, as older versions might have bugs or lack support for certain time zones.
This issue sometimes also caused due to different versions of Spark. With the spark version change, there also happens a change in the time API in Java 8's standard time zone, and thus should be dealt accordingly.
Thanks a lot. Yes, indeed this was caused by Spark version upgrade with was a direct result of glue job upgrade from version 2.0 to version 4.0. It appears within our code we are using a library which utilized time API in Java 8's standard time zone. We had to make slight modification in code to handle milliseconds and time zone offsets. It is working as expected now. Thanks again.
Relevant content
- Accepted Answerasked 3 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 2 years ago