Serverless job failure

Basically, each worker can have 1, 2, 4, 8, or 16 vCPUs/cores, in other words there is a limit set on the number of vCPUs for a worker that not valid for the core given as 7. Please refer this document for more information - https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/app-behavior.html

Hi,

Did you check that the value that you set spark.cores.max is greater than this value? If not, that may be the cause.

Best,

Didier