How do I restart the delivery of Application Load Balancer access logs to my Amazon S3 bucket?

Short description

Application Load Balancer access logs might not appear in your S3 bucket for the following reasons:

• You didn't turn on access logs in the Application Load Balancer configuration.
• The S3 bucket policy is missing a required permission.
• The AWS Identity and Access Management (IAM) role is missing, expired, or incorrectly configured.
• The bucket policy denies s3:PutObject for the elasticloadbalancing.amazonaws.com Application Load Balancer service principal.
• The S3 bucket name, path or prefix in the Application Load Balancer log settings is incorrect.
• Encryption settings interfere with the Application Load Balancer log.

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

Check your Application Load Balancer log configuration

To check whether you turned on access logs, run the following describe-load-balancers command:

aws elbv2 describe-load-balancers --load-balancer-arn arn-value --query "LoadBalancers[0].AccessLoggingEnabled" 

Note: Replace arn-value with your Application Load Balancer Amazon Resource Name (ARN).

If you didn't turn on access logs, then see Enable access logs for your Application Load Balancer.

To get S3 bucket configuration details, run the following describe-load-balancers command:

aws elbv2 describe-load-balancers --load-balancer-arn arn-value --query "LoadBalancers[0].AccessLoggingConfiguration" 

Note: Replace arn-value with your Application Load Balancer ARN.

If the S3BucketName and S3BucketPrefix don't match your target bucket, then update the access log configuration.

Check your S3 bucket policy

Check that the bucket policy grants s3:PutObject permission to the Application Load Balancer service principal.

If the permission is missing or denied, then add the following statement to the policy:

{ 
   "Version": "2012-10-17", 
    "Statement": [
       { 
         "Effect": "Allow", 
         "Principal": { 
            "Service": "logdelivery.elasticloadbalancing.amazonaws.com" 
       }, 
       "Action": "s3:PutObject", 
       "Resource": "arn:aws:s3::: amzn-s3-demo-bucket /prefix/AWSLogs/12345678901/*" 
     } 
   ] 
 } 

Note: Replace amzn-s3-demo-bucket with your bucket, prefix/AWSLogs with your path, and 12345678901 with your AWS account ID. Only load balancers from the specified account can write access logs to the S3 bucket.

Check your S3 bucket for FULL_CONTROL default permissions for the Application Load Balancer.

To use the AWS CLI, run the following get-bucket-acl command:

 aws s3api get-bucket-acl --bucket amzn-s3-demo-bucket

Note: Replace amzn-s3-demo-bucket with your bucket.

Check your IAM role permissions

Complete the following steps:

1. Open the IAM console.
2. In the navigation pane, choose Roles, and then select the AWSServiceRoleForElasticLoadBalancing service-linked role.
3. On the Permissions tab, under Permissions policies, check for the AmazonS3ReadOnlyAccess AWS managed policy or a custom policy that allows s3:PutObject.
4. Choose the Trust relationships tab to check that the role trusts the elasticloadbalancing.amazonaws.com service principal.
   Example trust policy statement:

   { 
       "Version": "2012-10-17", 
        "Statement": [ 
          { 
             "Effect": "Allow", 
             "Principal": { 
                "Service": "elasticloadbalancing.amazonaws.com" 
           }, 
           "Action": "sts:AssumeRole" 
         } 
       ] 
    } 
   

Check your encryption settings

Check that you have configured your server-side log encryption to use only Amazon S3-managed keys (SSE-S3).

Review metrics and logs

Take the following actions:

• Use the Amazon CloudWatch console to view the RequestCount or HTTPCode_ELB_5XX_Count metric.
  Note: For information about the metrics, see Application Load Balancer metrics. 
• Search AWS CloudTrail logs for PutObject events from elasticloadbalancing.amazonaws.com.

Reset the access log configuration

If there are no log configuration, permissions, or Application Load Balancer issues, then run the following modify-load-balancer-attributes command to reset the access log configuration:

aws elbv2 modify-load-balancer-attributes \ 
   --load-balancer-arn arn-value \ 
   --attributes Key=access_logs.s3_enabled,Value=true \ 
                Key=access_logs.s3_bucket,Value= amzn-s3-demo-bucket \ 
                Key=access_logs.s3_prefix,Value=prefix

Note: Replace arn-value with your Application Load Balancer ARN, amzn-s3-demo-bucket with your bucket, and prefix with your S3 bucket prefix.

Confirm that your S3 bucket receives logs 

Take the following actions:

• Send requests to your Application Load Balancer, wait 5-15 minutes for logs to appear, and then check your S3 bucket for new log files.
• Create a CloudWatch alarm for the NumberOfObjects metric, and check for nonzero values.
  Note: For information about the metric, see Amazon S3 daily storage metrics for buckets in CloudWatch.
• Use the s3-bucket-policy-grantee-check AWS Config managed rule to monitor bucket policy compliance.

Contact Support

If logs still don't appear, then contact AWS Support.

In your support case, provide the following details:

• Application Load Balancer ARN
• S3 bucket name and policy
• IAM role details
• Relevant CloudTrail or CloudWatch logs

Implement best practices

To prevent further issues with access log delivery, take the following actions:

• Review your S3 bucket policies and IAM roles quarterly to make sure that they remain accurate.
• Use AWS managed policies, such as AmazonS3ReadOnlyAccess, for Application Load Balancer logging roles.
• Turn on S3 Versioning on your S3 bucket so that logs don't get accidentally deleted.
• Use server-side encryption with Amazon S3 managed keys (SSE-S3) to secure log data and maintain compatibility with the Application Load Balancer.
• Set a S3 lifecycle configuration on your bucket to archive or delete logs after compliance retention periods.