Glue job keeps running but does not write results

0

Running a glue job to fetch records from Microsoft sql server but glue jobs keeps running and does not show any results. Job is scheduled with G.2X worker with 5 works with auto scheduling.

Logs:- 23/02/27 09:02:45 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

2개 답변
2

Can check the worker nodes to ensure that they have enough resources to run the job. The warning message in the logs suggests that the worker nodes may not have enough resources to execute the job.

답변함 일 년 전
0

The driver logs should tell you bit what the driver is doing, the warning could not be the cause.
It would be good to enable SparkUI logs and view then in a History Server to check what the driver is doing, has the driver started a Spark job/stage and doesn't get resources?

The most likely cause is that you have multiple stages in the job and the first stage is reading the data with a single task, SparkUI will tell you that. Also check this: https://docs.aws.amazon.com/glue/latest/dg/run-jdbc-parallel-read-job.html

profile pictureAWS
전문가
답변함 일 년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠