AWS Batch Compute Environment setup: If Minimum vCPUs is > 0 then instances keep restarting, how do I terminate them permanently?

0

When setting up a compute environment I set the minimum vCPUs to 64, and the desired vCPUs to 64 as well (maximum vCPUs is 1152). I didn't have a minimum set before, but now it keeps restarting instances all the time and I can't seem to get it to stop doing that. I've tried editing the minimum and desired vCPUs back to 0 and it still keeps restarting the instances after I terminate them and it doesn't save the changes that I made to the compute environment either. There aren't any jobs running that use this queue or compute environment either - why is it doing this and how do I get it to stop restarting the instances?

cthomas
asked 10 months ago267 views
3 Answers
0

If I change the max vCPU to 0, will the instance be launched repeatedly?
I think if the minimum vCPU is set to 0, the instance will not start when there is no job...
It may be a good idea to check again for jobs, etc. as described in the following document.
https://repost.aws/knowledge-center/batch-jobs-termination

profile picture
EXPERT
answered 10 months ago
0

I tried disabling the job queue and the compute environment (I even actually deleted the job queue) and the instances were still restarting on their own! It's like the compute environment was thinking that it always had to have the instances running no matter what, even when no jobs were running. I eventually managed to stop it somehow because it finally let me edit the compute environment - I think it was when I also set "Job execution timeout" to 0 but I'm not sure?

Either way it doesn't seem very intuitive. I'm hesitant to try setting any minimum vCPU value now in case I just turned it off by blind luck!

cthomas
answered 10 months ago
0

When Minimum vCPUs is set greater than zero, a Batch Compute Environment (Comp Env) will try to ensure that number of vCPUs are available at all times, and will spin up however many EC2 instances it needs to do so. The number and type of EC2s it spins up depends on the instance types/families you have allowed in the Comp Env configuration.

Once you've set Minimum vCPUs greater than zero, it can be difficult to set it back to zero because Batch gives an error message like "Manually scaling down compute environment is not supported". To set Minimum vCPUs back to zero, you need to: 1) wait for any jobs running to finish (or terminate them), 2) "disconnect" any job-queues that are configured to use the Comp Env by changing the job queue configuration, 3) disable the Comp Env, 4) terminate any instances still running in the Comp Env, and 5) change Minimum vCPUs to zero.

SE_Jeff
answered 6 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions