Hi.
I have set up an EMR Serverless application to run a simple Hello World spark job. I've configured everything properly and the job execution is successful.
Next, I've created a custom image according to the official docs:
https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/application-custom-image.html
However, once I've updated the application to use the custom image and submitted the same Spark job it fails. No logs are available and no clue about the error cause.
Please advise a possible solution.
Thanks.
If you're building on an M1 or M2 Mac, make sure you've added
--platform=linux/amd64
to theFROM
line in your Dockerfile or you've chosen arm64 architecture when you create your application.