AWS Glue is throwing this error - Failed to create any executor tasks

0

Running a Glue Job to write data to a database and read from S3 bucket. The job is failing with below error:

Py4JNetworkError: An error occurred while trying to connect to the Java server (127.0.0.1:38097) caused by Failed to create any executor tasks

Unable to find the cause. Have tried to print the dataframe, looks fine to me, failing on write statement. Please help.

preguntada hace 2 años4101 visualizaciones
1 Respuesta
0

The error that you are facing might be because your server is unreachable. This type of error generally occurs due to lack of free IP addresses in the subnet that is being used by your Glue job.

Whenever a Glue job is using a connection, Glue creates ENIs within the subnet so as to establish communication between your job and the data source. The executors use these ENIs to communicate among themselves. The number of ENIs created by Glue depends on the number of worker nodes deployed by your job. Each ENI requires an IP address. Therefore on a higher level, there must be enough number of free IP addresses available in your subnet such that each worker node gets one allocated and completes it's task successfully.

So please check the number of free IP addresses available in your subnet. You can refer this https://aws.amazon.com/premiumsupport/knowledge-center/glue-specified-subnet-free-addresses/ for more information.

profile pictureAWS
INGENIERO DE SOPORTE
Chaitu
respondido hace 2 años

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas