S3 static loop issue

0

Hi AWS,

I was going through a question from AWS CQ session i.e. An AWS Lambda function has been configured to create a thumbnail image whenever an object is created in a particular S3 bucket. However, it is getting stuck in an infinite loop. How this can be avoided ?.

I was assuming it happened because of the throttling, however the answer is Configure a Prefix on the Amazon S3 Event. Can you please give some more insights and share any link which covers this.

profile picture
Arjun
asked a year ago253 views
3 Answers
1

When placing the image back into the same bucket it came from, it needs to be differentiated. Therefore placing the thumbnail into a different folder.

Only monitor file creation in the folder the image is uploaded to. The lambda function needs to store the images it creates in a different folder (prefix)

See prefix on when creating event notifications

On the S3 bucket, Goto properties -> Event Notifications -> Create event notification Configure the prefix here (folder name)

https://docs.aws.amazon.com/AmazonS3/latest/userguide/notification-how-to-filtering.html

This means the lambda function will only trigger on the image upload and not the thumbnails created by lambda.

profile picture
EXPERT
answered a year ago
0

Hi, this is happening because if you don't take any precaution in the structure of your bucket and the way it is managed by the lamdbda, an initial image will trigger the lambda to generate a thumbnail written back to the bucket. The write of the thumbnail will again trigger the lambda to generate a thumbnail of the thumbnail ad infinitum.

So, you either need to have a folder in your bucket or some naming convention for the thumbnail allowing you to distinguish within the lambda if it was triggered by an initial image or a thumbnail. If lambda starts on a thumbail, it should just return without doing anything. That will stop recursivity.

profile pictureAWS
EXPERT
answered a year ago
  • This method triggers on false positives which is costly of compute resource and money. Prefix on event notifications on the bucket means the lambda function will only trigger on the correct images.

0

In short, avoid triggering the lambda from an s3 bucket, which is the same bucket accessed by the lambda to PUT the generated thumbnail, causing infinite loop (and costs).

profile picture
EXPERT
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions