Resource utilization exceed the limit as per billing in serverless

0

I run one application without pre-initialized capability in Emr serverless and thr explicit limit set to 40vCPUs, 300gb memory, 2000gb disk. After the every job execution, resources utilization and billing shows exceeded vCPUs limit for an instance sometimes shows 64.55 vCPU. May I know why this discrepancy and could someone please clear if you have experienced this before?

Thanks in advance

Vaas
已提問 6 個月前檢視次數 285 次
1 個回答
3
已接受的答案

Hello,

Basically, application levels limits are different from total Resource utilization in a job that showed in the console. Application limits helps control the total number of cores you can provision across multiple jobs running in an application concurrently whereas resource utilization is the total resources utilized in hours for billing calculation.

For an instance,Lets say, suppose I have 1 driver and 1 executor with 4vCPUs each, application limit with 10vCPUs and my job runs for 5hours, then the total resource vCPUs utilization will be approx. 80vCPUs hours. (4vCPUs driver + 4vCPUs executor = 8 * 5 hours = 40vCPUs).

AWS
支援工程師
已回答 6 個月前
profile picture
專家
已審閱 5 個月前
profile picture
專家
已審閱 6 個月前
  • It makes sense. Thanks for the clarification 👍

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南