Terminating EMR Serverless Applications Costs

0

Hi, I run EMR serverless jobs at the top of every hour. All the jobs are submitted to the same Application with no pre-initialized capacity. Is there any benefit from terminating the application between runs? There are about 30 minutes between runs. Any downsides in costs?

2 Answers
1
Accepted Answer

You are not charged for the EMR Serverless application itself. You are charged for:

  • Pre-initialized capacity while the application is STARTED.
  • On-demand workers during job runs

So no, no benefit from terminating the application. If you wanted to have quick job start time, you could configure pre-initialized capacity with a short auto-stop timeout (5 minutes), but that's only useful if immediate start time is important to your job. Without pre-initialized capacity, workers usually start within about a minute. You can find more pricing info on the EMR Serverless pricing page.

AWS
dacort
answered a year ago
AWS
SUPPORT ENGINEER
reviewed 2 days ago
-1

EMR of 2 components in regards to pricing. They have the EC2 compute cost and the EMR cost [1]. Both of these have a 1 min minimum. If you are running jobs for 30 min every hour, then turning off the cluster after 30 min could save you 1/2 your costs.

I also looked into Savings Plans (SP) [2] or Reserved Instances (RI) [3], these can save you more than 50% of the EC2 costs. There is no SP or RI for the EMR costs, but those costs are generally about 25% of the EC2 cost. So if your selected SP or RI can save you 75% (50% of only running half the time + 25% of the EMR cost, it would actually be 25% for the 1/2 hour the cluster is idle) of the costs, then it would be worthwhile to go with this option.

Finally, Task nodes (and ONLY Task nodes) can also be run on spot [4]. The primary and core nodes would then be run as on-demand or you can purchase an SP/RI. This would require having a large enough instance pool [5], but running spot instances can save you at a higher percentage and then you can shut down the cluster when you're not using it.

The benefit of having the cluster always on (and so you should purchase SP/RIs) is that you don't have to worry about gathering instances during the spin up of the cluster, they are always available to you. And this also takes into account the possibility that your workload may fail and need to be run again or takes longer than the 30 min you have allocated.

[1] https://aws.amazon.com/emr/pricing/ [2] https://aws.amazon.com/savingsplans/ [3] https://aws.amazon.com/ec2/pricing/reserved-instances/ [4] https://aws.amazon.com/ec2/spot/ [5] https://docs.aws.amazon.com/whitepapers/latest/cost-optimization-leveraging-ec2-spot-instances/spot-best-practices.html

profile pictureAWS
EXPERT
answered a year ago
  • Hi, Thank you for your response although I was asking specifically asking about EMR Serverless which I don't see any notion of the EC2 side. Only CPU, Memory and Disk

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions