Help improve AWS Support Official channel in re:Post and share your experience - complete a quick three-question survey to earn a re:Post badge!
How can I audit deleted or missing objects from my Amazon S3 bucket?
There's an object or file that's missing from my Amazon Simple Storage Service (Amazon S3) bucket. I want to find information about how the object or file was deleted, and prevent future accidental deletions.
Resolution
To find out how an S3 object was deleted, you can review either server access logs or AWS CloudTrail logs.
Note: You must turn on logging for the bucket before the deletion event occurs. You receive logs only for events that occurred after you turned on logging.
Server access logs
Server access logs track S3 operations manually performed or as part of a lifecycle configuration. To turn on server access logging, see Enabling Amazon S3 server access logging. For more information on how to analyze server access logs, see How do I analyze my Amazon S3 server access logs using Athena?
CloudTrail logs
CloudTrail logs can track object-level data events in an S3 bucket, such as GetObject, DeleteObject, and PutObject. By default, CloudTrail records bucket-level events. To turn on CloudTrail logging for object-level events, see Enabling CloudTrail event logging for S3 buckets and objects. For more information on how to find specific events, see Why aren't Amazon S3 object-level API actions appearing in my CloudTrail Event history?
Note: Because object-level logging incurs additional charges, make sure to review the pricing for CloudTrail data events.
To prevent future accidental deletions, it's a best practice to use one of the following features:
- Turn on versioning to keep historical versions of an object.
- Turn on Cross-Region Replication of objects.
- Turn on MFA delete to require multi-factor authentication (MFA) when you delete an object version.
- Use AWS Backup to restore an object or file.
- Use S3 Object Lock to prevent the deletion of an object.
This article implies that S3 server access logging and CloudTrail data event logging would be equally effective at tracking object deletions. That is not correct. CloudTrail data events don't include the object keys or version IDs at all, when objects are deleted as part of a batch delete operation with the DeleteObjects API. I believe the management console uses DeleteObjects even to delete a single object or object version, making this a very typical situation.
Keys not being logged for batch deletes is mentioned as a limitation in this comparison table in documentation: https://docs.aws.amazon.com/AmazonS3/latest/userguide/logging-with-S3.html ("Logging of keys in a batch delete operation").
The solution is to use S3 server access logging to audit all object deletions, regardless of the method used to trigger each deletion, and not to rely solely on CloudTrail data event logging.
Relevant content
- asked 10 months ago
- asked 4 years ago
- asked a year ago
- asked a year ago
- asked a year ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 2 years ago
