- 最新
- 最多得票
- 最多評論
Hello Muthukumar,
Above error stack looks generic. If the EMR-S job failed for any reason, above exception would be captured in the MWAA. Can you please take a look at the failed job in the EMR-S application ? If the job submission fails for any reason, then it should be an issue with spark-submit configuration including the arguments passed. Please validate if they are correct.
If the job failed after created the sparkContext then, the stderr log link would be generated in the job and also make sure the logging is configured to s3.
You can also submit the job directly to EMR-S and test if they are working fine.
Hi Yogesh,
The job is working fine if submitted manually as well as from airflow. Only in the above scenario, the job submitted from airflow received a job run ID as well, but no status except("the job is pending scheduling") in EMR serverless. both stdout and stderr are not opening.
相關內容
- 已提問 7 個月前
Hello Muthukumar,
Thanks for writing back. If the issue is not producible again, then it might be intermittently occurred. If you share me the Job-ID/Application-ID from existing AWS Account Support case, I can take a look from the backend.
Best Regards, Yokesh