Resource utilization exceed the limit as per billing in serverless

0

I run one application without pre-initialized capability in Emr serverless and thr explicit limit set to 40vCPUs, 300gb memory, 2000gb disk. After the every job execution, resources utilization and billing shows exceeded vCPUs limit for an instance sometimes shows 64.55 vCPU. May I know why this discrepancy and could someone please clear if you have experienced this before?

Thanks in advance

Vaas
posta 6 mesi fa287 visualizzazioni
1 Risposta
3
Risposta accettata

Hello,

Basically, application levels limits are different from total Resource utilization in a job that showed in the console. Application limits helps control the total number of cores you can provision across multiple jobs running in an application concurrently whereas resource utilization is the total resources utilized in hours for billing calculation.

For an instance,Lets say, suppose I have 1 driver and 1 executor with 4vCPUs each, application limit with 10vCPUs and my job runs for 5hours, then the total resource vCPUs utilization will be approx. 80vCPUs hours. (4vCPUs driver + 4vCPUs executor = 8 * 5 hours = 40vCPUs).

AWS
TECNICO DI SUPPORTO
con risposta 6 mesi fa
profile picture
ESPERTO
verificato 5 mesi fa
profile picture
ESPERTO
verificato 6 mesi fa
  • It makes sense. Thanks for the clarification 👍

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande