AWS Glue is throwing this error - Failed to create any executor tasks

0

Running a Glue Job to write data to a database and read from S3 bucket. The job is failing with below error:

Py4JNetworkError: An error occurred while trying to connect to the Java server (127.0.0.1:38097) caused by Failed to create any executor tasks

Unable to find the cause. Have tried to print the dataframe, looks fine to me, failing on write statement. Please help.

posta 2 anni fa4101 visualizzazioni
1 Risposta
0

The error that you are facing might be because your server is unreachable. This type of error generally occurs due to lack of free IP addresses in the subnet that is being used by your Glue job.

Whenever a Glue job is using a connection, Glue creates ENIs within the subnet so as to establish communication between your job and the data source. The executors use these ENIs to communicate among themselves. The number of ENIs created by Glue depends on the number of worker nodes deployed by your job. Each ENI requires an IP address. Therefore on a higher level, there must be enough number of free IP addresses available in your subnet such that each worker node gets one allocated and completes it's task successfully.

So please check the number of free IP addresses available in your subnet. You can refer this https://aws.amazon.com/premiumsupport/knowledge-center/glue-specified-subnet-free-addresses/ for more information.

profile pictureAWS
TECNICO DI SUPPORTO
Chaitu
con risposta 2 anni fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande