AWS S3 Advanced metrics and recommendations

0

Hi,

Recently we are noticing huge spike in s3 api requests. In order to find out the top buckets which are getting the api requests, we need to enable the Advanced metrics and recommendations in default account dashboard in storage lens. If we enable this setting, will we able to disable and set it back for free metrics?

Is there any other approach to achieve this? Our goal is to find out which buckets are getting the most hits for tier-1 and tier-2 requests and we need to from which aws service are calling those requests.

asked 2 months ago120 views
3 Answers
1

Hello.

Since the Storage Lens dashboard can be deleted, I think it is possible to disable or delete it later.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage_lens_console_disabling_deleting.html

Is there any other approach to achieve this?

By recording data events with CloudTrail, you can keep a record of the S3 API.
I think it is possible to determine which APIs were executed the most from that log.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudtrail-logging-s3-info.html

profile picture
EXPERT
answered 2 months ago
profile picture
EXPERT
reviewed 2 months ago
1

Hi,

You may want to analyze the so-called Server Access logs: see https://docs.aws.amazon.com/AmazonS3/latest/userguide/ServerLogs.html to set up those logs.

Then, this KC article will tell you how to analyze them: https://repost.aws/knowledge-center/analyze-logs-athena

Best,

Didier

profile pictureAWS
EXPERT
answered 2 months ago
profile picture
EXPERT
reviewed 2 months ago
0

Hello,

To identify top buckets receiving S3 API requests and their originating services:

  • Enable Advanced Metrics: This provides detailed data on S3 usage.
  • Analyze Metrics: Look for buckets with high request volumes and identify the services making those requests.
  • Use CloudTrail: For more granular details on individual API calls.
  • Create Custom Metrics: If needed, to track specific data points.

will we able to disable and set it back for free metrics?

  • Yes, you can easily disable Advanced Metrics and Recommendations in Storage Lens and revert back to the free metrics level at any time. There are no additional costs or penalties associated with switching between these options.

Is there any other approach to achieve this?

  • CloudWatch Logs: You can enable CloudWatch Logs for your S3 buckets to capture detailed information about API calls.
  • CloudTrail: CloudTrail is another option that can provide valuable insights into S3 API activity. It records all API calls made to your S3 resources, allowing you to analyze requestors, parameters, and responses.
  • Custom Metrics: If you need more specific metrics tailored to your use case, you can create custom metrics using Amazon CloudWatch. This gives you flexibility to track and analyze data points that are not provided by default metrics.

While these approaches can provide valuable information, Advanced Metrics in Storage Lens offers a more comprehensive and integrated solution for analyzing S3 usage.

To identify top buckets receiving tier-1 and tier-2 requests and their originating AWS services:

  • Enable Advanced Metrics in Storage Lens: This provides detailed metrics on S3 usage, including request types and originating services.
  • Analyze Metrics: Look for buckets with high request volumes for tier-1 and tier-2 requests.
  • Identify Originating Services: Use the metrics to determine which AWS services are making these requests.
profile picture
EXPERT
answered 2 months ago
profile picture
EXPERT
reviewed 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions