Does AWS Transfer Family handle file 'rename' events?

0

Greetings:

We're currently operating a Transfer Family server for the usual SFTP endpoint scenarios. As our customers upload their files to Transfer Family, we use a custom workflow to trigger a Lambda which starts the 'processing' of that file. This all currently works just fine, we're happy with it.

One edge case we're running into is if a customer uploads a file and then quickly renames the file via their SFTP client. In most cases, these rename operations happen within ~10 seconds of the original file upload. As mentioned above, the Lambda that kicks off the processing of this file still refers to the pre-rename file path, which now due to the rename, isn't a valid path anymore in S3.

We've done some testing and found that the custom Transfer Family workflow that we have isn't triggered at all for the 'file rename' event, meaning we never see a 'rename' event in the custom workflow which is our only way to trigger file processing currently. From the customer's point of view, their uploaded file is never processed at all. From a developer's point of view, Transfer Family has allowed the customer to do something that we can't action against because we're never notified about it using existing workflows. Of course, a solution could exist and we haven't found it, I would be excited to be wrong. :)

The only record that we have that even registers the file rename event is via the CloudWatch logs that we have set up to receive from Transfer Family. The event data looks like this in those logs

23423485a3fead4834763451c0ee8c02a.ed234gfds3d2373 RENAME Path=/foobar/myfile.csv NewPath=/foobar/myfile_renamed.csv

My question is -- is there a product solution to this? Are file renames a use case in Transfer Family? After researching, it seems like no. I might be able to add something to parse CloudWatch logs to look for RENAME in log entries above, but that seems .. hacky. Has anyone created a suitable solution to this problem recently? Thanks for reading, appreciate your time.

melgart
asked 2 months ago425 views
1 Answer
1

Hello,

AWS Transfer Family does not inherently support file renaming events. However, a viable solution involves integrating file renaming logic into a Lambda function triggered by Transfer Family events through Amazon EventBridge. For instance, upon completion of a file upload, Lambda could be programmed to duplicate the object to a new key or name in S3.

Thanks

answered 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions