S3 Batch Operations Job stays Active


S3 Batch Operations jobs stays "Active" for ever. It has accessed the manifest and shows the number of objects as expected. But after that it doesn't trigger the lambda nor fails with any response. Any help much appreciated

  • Hello,

    Thank you for reaching out. Can you provide your region and job id?

    Thank you!

asked 2 years ago1279 views
2 Answers

A batch job performs a specified operation on every object that is included in its manifest. A manifest lists the objects that you want a batch job to process and it is stored as an object in a bucket. After you create a job, Amazon S3 processes the list of objects in the manifest and runs the specified operation against each object.

When a S3 batch job is "active" it means Amazon S3 is performing the requested operation on the objects listed in the manifest. While a job is Active, you can monitor its progress using the Amazon S3 console or the DescribeJob operation through the REST API, AWS CLI, or AWS SDKs.

The performance/speed of any particular Batch Operations job will depend on a variety of factors, including :

 * The number of objects in the manifest
 * The type of operation (Glacier restore, copy, Lambda function, etc)
 * The number of active jobs your account
 * The number of enqueued jobs for your account
 * Other traffic to the source and/or destination bucket
 * The size of the objects

You can refer to the following link for information on troubleshooting S3 batch jobs: https://aws.amazon.com/premiumsupport/knowledge-center/s3-troubleshoot-batch-operations/

If you need additional support, then you can also open a case with AWS Premium Support to review the details of this particular job.

answered 2 years ago

@ron_h, @Sathya, Thanks for your responses.

Following jobs were my jobs of concern. Though, now the jobs say they completed in ~5 mins after being created, it was not like that on 12th. https://s3.console.aws.amazon.com/s3/jobs/aa55146f-21d2-4c06-801e-29908cd20be5?region=eu-west-2 https://s3.console.aws.amazon.com/s3/jobs/9ba4123d-2f1f-479f-b4b5-cb4bf94ef2b5?region=eu-west-2

As the subsequent jobs, after i had to fix something in my lambda function, moved to completion as expected, may be the above two jobs didnt get updated in GUI in time.

Bottom line, things are ok now. We can close the ticket from my side.

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions