Exporting Cloudwatch log group To S3 is failing

0

I am trying to export data to s3, but its failing for the following error

Could not create export task. GetBucketAcl call on the given bucket failed. Please check if CloudWatch Logs has been granted permission to perform this operation.

This is the step I am doing From Cloudwatch Selected log Group Selected the specific log I want to export to S3 Selected Action Export Data to Amazon S3 and I get the error

Could not create export task. GetBucketAcl call on the given bucket failed. Please check if CloudWatch Logs has been granted permission to perform this operation.

I have all the right permission including GetBucketACL to the bucket policy . I am really not sure how to fix this error . ofcourse thats the only error and not specific to what the issue is . """"Please check if CloudWatch Logs has been granted permission to perform this operation." where do I check if Cloudwatch logs has been granted permission .

asked 7 months ago1519 views
4 Answers
0
Accepted Answer

I tried everything possible . No matter what I am doing I am still getting the same error

OK, I'll give you a sample that actually works for me. This sample configuration might give you some insights.

AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: Export CloudWatch Logs to S3 Bucket

Resources:
  # S3
  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - BucketKeyEnabled: true
            ServerSideEncryptionByDefault:
              SSEAlgorithm: AES256

  S3BucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket:
        Ref: S3Bucket
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Sid: S3BucketReadPermissions
            Effect: Allow
            Principal:
              Service:
                Fn::Sub: logs.${AWS::Region}.amazonaws.com
            Action:
              - s3:GetBucketAcl
            Resource:
              - Fn::GetAtt: S3Bucket.Arn
            Condition:
              StringEquals:
                aws:SourceAccount:
                  Ref: AWS::AccountId
              ArnLike:
                aws:SourceArn:
                  Fn::GetAtt: HelloWorldFunctionLogGroup.Arn
          - Sid: S3ObjectWritePermissions
            Effect: Allow
            Principal:
              Service:
                Fn::Sub: logs.${AWS::Region}.amazonaws.com
            Action:
              - s3:PutObject
            Resource:
              - Fn::Sub: ${S3Bucket.Arn}/*
            Condition:
              StringEquals:
                s3:x-amz-acl: bucket-owner-full-control
                aws:SourceAccount:
                  Ref: AWS::AccountId
              ArnLike:
                aws:SourceArn:
                  Fn::GetAtt: HelloWorldFunctionLogGroup.Arn

  # Lambda
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      Description: HelloWorld function
      Handler: index.handler
      Runtime: nodejs18.x
      Architectures:
        - arm64
      InlineCode: |
        exports.handler = async event => {
          console.log(JSON.stringify(event, null, 2));
          return { "message": "Hello World!" };
        };

  HelloWorldFunctionLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName:
        Fn::Sub: /aws/lambda/${HelloWorldFunction}

Assuming that you have administrator privileges and not restricted by your organization (e.g., SCP), here are the steps to reproduce my config:

  1. Save the configuration above in YAML file.
  2. Create a CloudFormation stack from the YAML file you saved.
  3. Check the resources the CloudFormation stack has created. It should contain an S3 bucket, a Lambda Function, and a CloudWatch Log Group.
  4. Go to the Lambda Console and invoke the new created Lambda function (e.g., stackname-HelloWorldFunction-xxxxyyyyzzzz).
  5. Go to the CloudWatch Console and export Lambda execution logs from the new created CloudWatch Log Group (e.g., /aws/lambda/stackname-HelloWorldFunction-xxxxyyyyzzzz) to the new created S3 bucket (stackname-s3bucket-xxxxyyyyzzzz).
  6. After successful test, review the configurations (e.g., bucket policy) and compare with your current configurations.
  7. Empty the bucket and delete the CloudFormation stack.
profile picture
HS
answered 7 months ago
0

Hi,

I have all the right permission including GetBucketACL to the bucket policy

You should be able to export your logs to S3 if you have set the correct bucket policy. Check your bucket policy carefully for the GetBucketAcl statement.

where do I check if Cloudwatch logs has been granted permission

If you cannot export logs from your log group to your bucket, then your log group has not granted proper permissions.

profile picture
HS
answered 7 months ago
0

Follow the instructions here: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3ExportTasksConsole.html

Highlights:

  1. Setup a bucket - obviously already done.
  2. Set up access permissions: your IAM user or if you're federating in, an IAM role you're using to access the console must have the following rights:
  • logs:CreateExportTask
  • logs:CancelExportTask
  • logs:DescribeExportTasks
  • logs:DescribeLogStreams
  • logs:DescribeLogGroups
  1. Setup bucket permissions: copy and paste the policy in your bucket and make sure you properly update all the red text to reflect your bucket name and the region it is hosted in.

Side note:

  • "arn:aws:s3:::my-exported-logs" - refers to the bucket itself
  • "arn:aws:s3:::my-exported-logs/*" - refers to the objects in the bucket.

Using KMS? Perform updates on your KMS key using provided steps.

AWS
LondonX
answered 7 months ago
0

I tried everything possible . No matter what I am doing I am still getting the same error

Could not create export task. GetBucketAcl call on the given bucket failed. Please check if CloudWatch Logs has been granted permission to perform this operation.

answered 7 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions