Cross-region copy of DocumentDB backup with Backup Service not working

Question

Hey guys,

I want to backup my DocumentDB cluster which is running in region eu-central-1 to region eu-north-1 (same account) using the AWS Backup service.
I created a backup plan with a copy operation to the backup vault in region eu-north-1. The backup in region eu-central-1 is working fine but the copy jobs are not even starting so I don't have any error messages that I could share.

These are my configurations:

**eu-central-1**

1. DocumentDB cluster: configured with multi-region KMS key
2. KMS key policy:

```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Enable IAM User Permissions",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::ACCOUNT_ID:root"
      },
      "Action": "kms:*",
      "Resource": "*"
    },
    {
      "Sid": "Allow use of the key",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::ACCOUNT_ID:root"
        ]
      },
      "Action": [
        "kms:DescribeKey",
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:ReEncrypt*",
        "kms:GenerateDataKey"
      ],
      "Resource": "*"
    },
    {
      "Sid": "Allow attachment of persistent resources",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::ACCOUNT_ID:root"
        ]
      },
      "Action": [
        "kms:CreateGrant",
        "kms:ListGrants",
        "kms:RevokeGrant"
      ],
      "Resource": "*",
      "Condition": {
        "Bool": {
          "kms:GrantIsForAWSResource": "true"
        }
      }
    },
    {
      "Sid": "Allow AWS Backup cross-region operations",
      "Effect": "Allow",
      "Principal": {
        "Service": "backup.amazonaws.com"
      },
      "Action": [
        "kms:Decrypt",
        "kms:DescribeKey",
        "kms:Encrypt",
        "kms:GenerateDataKey*",
        "kms:ReEncrypt*"
      ],
      "Resource": "*"
    }
  ]
}
```

3. IAM role policy (attached to Backup rule)

```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "backup:CopyIntoBackupVault",
                "backup:StartBackupJob",
                "backup:GetBackupVaultAccessPolicy",
                "backup:StartCopyJob",
                "backup:ListBackupVaults",
                "backup:GetRecoveryPoint"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:backup:eu-north-1:ACCOUNT_ID:backup-vault/*"
            ]
        },
        {
            "Action": [
                "backup:GetRecoveryPoint",
                "backup:ListRecoveryPointsByBackupVault",
                "backup:CopyIntoBackupVault",
                "backup:StartCopyJob"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:backup:eu-central-1:ACCOUNT_ID:recovery-point/*",
                "arn:aws:backup:eu-north-1:ACCOUNT_ID:backup-vault/*",
                "arn:aws:backup:eu-central-1:ACCOUNT_ID:recovery-point:*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kms:Decrypt",
                "kms:Encrypt",
                "kms:DescribeKey",
                "kms:CreateGrant",
                "kms:GenerateDataKey*",
                "kms:ReEncryptFrom",
                "kms:ReEncryptTo"
            ],
            "Resource": [
                "arn:aws:kms:eu-central-1:ACCOUNT_ID:key/*",
                "arn:aws:kms:eu-north-1:ACCOUNT_ID:key/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "rds:DescribeDBClusterSnapshots"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "kms:Encrypt",
                "kms:GenerateDataKey*"
            ],
            "Resource": "arn:aws:kms:eu-central-1:ACCOUNT_ID:key/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "kms:CreateGrant"
            ],
            "Resource": "arn:aws:kms:eu-central-1:ACCOUNT_ID:key/*",
            "Condition": {
                "Bool": {
                    "kms:GrantIsForAWSResource": "true"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:DescribeKey",
                "kms:CreateGrant",
                "kms:GenerateDataKey*"
            ],
            "Resource": "arn:aws:kms:eu-north-1:ACCOUNT_ID:key/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "kms:CreateGrant"
            ],
            "Resource": "arn:aws:kms:eu-north-1:ACCOUNT_ID:key/*",
            "Condition": {
                "Bool": {
                    "kms:GrantIsForAWSResource": "true"
                }
            }
        }
    ]
}
```

as well as managed AWS policies AWSBackupServiceRolePolicyForBackup and AWSBackupServiceRolePolicyForRestores

**eu-north-1**

1. Backup vault policy

```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Allow to copy into the vault",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "backup:CopyIntoBackupVault",
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:PrincipalOrgID": "ORG_ID"
                }
            }
        }
    ]
}
```

2. KMS key (replicated from region eu-central-1)

```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Enable IAM User Permissions",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::ACCOUNT_ID:root"
      },
      "Action": "kms:*",
      "Resource": "*"
    },
    {
      "Sid": "Allow use of the key",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::ACCOUNT_ID:root"
        ]
      },
      "Action": [
        "kms:DescribeKey",
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:ReEncrypt*",
        "kms:GenerateDataKey"
      ],
      "Resource": "*"
    },
    {
      "Sid": "Allow attachment of persistent resources",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::ACCOUNT_ID:root"
        ]
      },
      "Action": [
        "kms:CreateGrant",
        "kms:ListGrants",
        "kms:RevokeGrant"
      ],
      "Resource": "*",
      "Condition": {
        "Bool": {
          "kms:GrantIsForAWSResource": "true"
        }
      }
    },
    {
      "Sid": "Allow AWS Backup operations in Stockholm",
      "Effect": "Allow",
      "Principal": {
        "Service": "backup.amazonaws.com"
      },
      "Action": [
        "kms:Decrypt",
        "kms:DescribeKey",
        "kms:Encrypt",
        "kms:GenerateDataKey*",
        "kms:ReEncrypt*"
      ],
      "Resource": "*"
    }
  ]
}
```

Both backup vaults in both regions are encrypted with the KMS key.

Do you see any missing stuff in these configurations?

Best regards,
Mark

re:Post Agent · Answer

Based on your configuration details, there are a few potential issues that could be preventing your cross-region DocumentDB backup copies from starting:

1. DocumentDB and RDS backup limitations: AWS Backup has some specific limitations when working with database services. For DocumentDB (which uses the RDS backup infrastructure), there may be issues with cross-region copying when using certain encryption configurations.

2. IAM Role permissions: While your IAM policy looks comprehensive, you might be missing specific permissions needed for DocumentDB backups. The AWS Backup service role needs appropriate permissions to work with DocumentDB clusters specifically.

3. KMS key configuration: Although you've configured multi-region KMS keys, there might be an issue with how AWS Backup is interacting with these keys. For cross-region copies, both the source and destination KMS keys need specific permissions.

4. Copy job visibility: It's possible that the copy jobs are failing silently. Check the AWS Backup console's "Jobs dashboard" in both regions to see if there are any failed jobs that might provide more information.

5. Backup vault configuration: Your backup vault policy in eu-north-1 uses a condition based on Organization ID. Make sure your account is actually part of that organization and that the ID is correct.

To troubleshoot this issue:

1. Check if there are any failed copy jobs in the AWS Backup console's Jobs dashboard.

2. Verify that your IAM role has explicit permissions for DocumentDB operations (you have RDS permissions, but might need specific DocumentDB ones).

3. Try enabling CloudWatch logs for AWS Backup to get more detailed information about what's happening when the backup copy should be triggered.

4. Consider testing with a simpler configuration first (like using the default AWS managed KMS keys) to isolate whether the issue is related to your custom KMS key setup.

If none of these suggestions resolve the issue, you might need to contact AWS Support for more specific guidance on DocumentDB cross-region backup copies.  
**Sources**  
[Take Rds Snapshot through aws backup & trasfer to another region & Copy button is disabled in another region | AWS re:Post](https://repost.aws/questions/QU0qJtx7UGRu-nnY6qJEKJwg/take-rds-snapshot-through-aws-backup-trasfer-to-another-region-copy-button-is-disabled-in-another-region)  
[S3 Backup Copy to destination no errors yet nothing copied | AWS re:Post](https://repost.aws/questions/QUSjpY5xjmTfWQZscr0doC1Q/s3-backup-copy-to-destination-no-errors-yet-nothing-copied)  
[Troubleshooting AWS Backup - AWS Backup](https://docs.aws.amazon.com/aws-backup/latest/devguide/troubleshooting.html)  
[AWS Backup console dashboards - AWS Backup](https://docs.aws.amazon.com/aws-backup/latest/devguide/backup-dashboards.html)

Cross-region copy of DocumentDB backup with Backup Service not working

Add your answer

Relevant content