save logs of each outbound data transfer response from cloudfront to s3


I need to do this to charge my clients for data usage, so for each request I would need to get the request header with the output data transfer response and save it to s3

1 Answer

Hi there. I provided some options in your other question, but now I think I understand your request better.

My first question is would be is it possible to have your clients traffic on different distributions? If so, the built-in usage report might give you what you need.

If not, then I would recommend option 2 that I specified in my other answer, sending log data to AWS OpenSearch. This would allow you to use tools like Kibana to parse the logs and create custom dashboards. You can use the sc-bytes field to get the data transferred, and then correlate with other fields like cs-host (if filtering by host) or cs-uri-step (if filtering by URI), etc. More info on fields available here.

Here is a good blog post that gives an overview of how to set this up. Just keep in mind that the ElasticSearch service was recently renamed to AWS OpenSearch in 2021, but the blog post is still relevant.

Another option that you could look into is sending the standard CloudFront logs to CloudWatch. I have not used this option as much, so I don't know if it will get you exactly what you need, but it might be worth investigating. Here is another blog post that might help.

profile pictureAWS
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions