AWS Glue is throwing this error - Failed to create any executor tasks

0

Running a Glue Job to write data to a database and read from S3 bucket. The job is failing with below error:

Py4JNetworkError: An error occurred while trying to connect to the Java server (127.0.0.1:38097) caused by Failed to create any executor tasks

Unable to find the cause. Have tried to print the dataframe, looks fine to me, failing on write statement. Please help.

已提問 2 年前檢視次數 4101 次
1 個回答
0

The error that you are facing might be because your server is unreachable. This type of error generally occurs due to lack of free IP addresses in the subnet that is being used by your Glue job.

Whenever a Glue job is using a connection, Glue creates ENIs within the subnet so as to establish communication between your job and the data source. The executors use these ENIs to communicate among themselves. The number of ENIs created by Glue depends on the number of worker nodes deployed by your job. Each ENI requires an IP address. Therefore on a higher level, there must be enough number of free IP addresses available in your subnet such that each worker node gets one allocated and completes it's task successfully.

So please check the number of free IP addresses available in your subnet. You can refer this https://aws.amazon.com/premiumsupport/knowledge-center/glue-specified-subnet-free-addresses/ for more information.

profile pictureAWS
支援工程師
Chaitu
已回答 2 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南