AWS Glue is throwing this error - Failed to create any executor tasks

0

Running a Glue Job to write data to a database and read from S3 bucket. The job is failing with below error:

Py4JNetworkError: An error occurred while trying to connect to the Java server (127.0.0.1:38097) caused by Failed to create any executor tasks

Unable to find the cause. Have tried to print the dataframe, looks fine to me, failing on write statement. Please help.

gefragt vor 2 Jahren4101 Aufrufe
1 Antwort
0

The error that you are facing might be because your server is unreachable. This type of error generally occurs due to lack of free IP addresses in the subnet that is being used by your Glue job.

Whenever a Glue job is using a connection, Glue creates ENIs within the subnet so as to establish communication between your job and the data source. The executors use these ENIs to communicate among themselves. The number of ENIs created by Glue depends on the number of worker nodes deployed by your job. Each ENI requires an IP address. Therefore on a higher level, there must be enough number of free IP addresses available in your subnet such that each worker node gets one allocated and completes it's task successfully.

So please check the number of free IP addresses available in your subnet. You can refer this https://aws.amazon.com/premiumsupport/knowledge-center/glue-specified-subnet-free-addresses/ for more information.

profile pictureAWS
SUPPORT-TECHNIKER
Chaitu
beantwortet vor 2 Jahren

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen