DataSync location access test failed: could not perform S3:ListObjectsV2

Question

> Resource handler returned message:
"Invalid request provided: DataSync location access test failed: could not perform S3:ListObjectsV2 on bucket *my_bucket* Access denied. Ensure bucket access role has S3:ListBucket permission.

I am trying to create an S3 location for a DataSync Task and consistently running into the above error despite following [AWS best practices ](https://docs.aws.amazon.com/datasync/latest/userguide/create-s3-location.html). **Note, S3:ListBucket is already given as a permission.**

The error only occurs on "fresh" deploys where the stack is not already previously deployed successfully. My current workaround is to deploy granting the role S3 full access and then redeploy with the below policy. Below is the relevant portion of the python cdk code I am using.

Is this an AWS DataSync problem? Or a cdk bug?

```
def datasync_s3_access_role(bucket: s3.Bucket, role_name_suffix: str) -> iam.Role:
            
            # create bucket level policy
            datasync_s3_bucket_policy =iam.PolicyStatement(
                actions=[
                    "s3:GetBucketLocation",
                    "s3:ListBucket",
                    "s3:ListBucketMultipartUploads"
                ],
                resources=[bucket.bucket_arn],
                conditions={"StringEquals": {
                                    "aws:ResourceAccount": [self.account_full]
                    }}
                )
            
            # create object level policy
            datasync_s3_object_policy=iam.PolicyStatement(
                actions=[                
                    "s3:AbortMultipartUpload",
                    "s3:DeleteObject",
                    "s3:GetObject",
                    "s3:GetObjectTagging",
                    "s3:GetObjectVersion",
                    "s3:GetObjectVersionTagging",
                    "s3:ListMultipartUploadParts",
                    "s3:PutObject",
                    "s3:PutObjectTagging",
                ],
                resources=[f"{bucket.bucket_arn}/*"],
                conditions={"StringEquals": {
                        "aws:ResourceAccount": [self.account_full]
                    }}
            )
            
            # Create IAM role and policies for DataSync S3 access
            datasync_s3_access_role = iam.Role(self, 
                                                f"DataSyncRole-{role_name_suffix}-{env_config}",
                                                assumed_by=iam.ServicePrincipal("datasync.amazonaws.com"),
                                                role_name=f"DataSyncRole-{role_name_suffix}-{env_config}",)
            
            datasync_s3_access_role.add_to_policy(datasync_s3_bucket_policy)
            datasync_s3_access_role.add_to_policy(datasync_s3_object_policy)
            
            return datasync_s3_access_role

datasync_s3_access_role_oci_to_aws = datasync_s3_access_role(oci_to_aws_bucket, 'oci-to-aws')
        datasync_s3_access_role_aws_to_oci = datasync_s3_access_role(aws_to_oci_bucket, 'aws-to-oci')
```

Riku_Kobayashi · Answer

Hello.

Can I configure it by operating from the management console instead of CDK?

I did a little search for issues on GitHub, but couldn't find any similar reports.  
Since it may be a bug, why not report it as an issue?  
https://github.com/aws/aws-cdk/issues

You may be able to check a more detailed error in "ListBuckets" from CloudTrail's event history.  
https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events-console.html

Leo K · Answer

I also can't see anything wrong with the policy definition. `s3:ListBucket` isn't an API but a permission, which permits several APIs, such as `ListObjects`, `ListObjectsV2`, and `HeadBucket`. They are considered as data operations, so they won't show in the default CloudTrail trail only containing management events.

The only thing I can see that would obviously refuse the operation but permit it with the AmazonS3FullAccess policy attached would be if the bucket resided in a different AWS account, since the policy used the `aws:ResourceAccount` condition key to ensure it's in the local account, while AmazonS3FullAccess would allow access to any target account.

The best thing to do would be to enable data event logging for S3 in CloudTrail (https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudtrail-logging.html). The logs would easily show exactly what is attempted details about the context in which it's being blocked.

You could also extract the policy from the IAM console after deploying it with CDK to confirm that the output is as expected, such as containing the correct bucket ARN and account ID. These are not very likely to be wrong, but it's a simple sanity check to make.

DataSync location access test failed: could not perform S3:ListObjectsV2

Relevant content