Cloudformation failing to create RDS encrypted database with created KMS Key "Encrypted instances require a valid KMS key ID"

0

I'm creating my structure using CloudFormation but I'm stuck with an error that doesn't make sense. To summarize, I'm creating a KMS Key and using it in the Stack itself to configure a Postgres Encrypted RDS.

The key is always created successfully, in the database I use DependsOn to make sure that Cloudformation is following the sequence.

The process runs successfully until the last step where it waits for an ok to create the RDS, I see the instance successfully in the console with the key created correctly referenced, but after a few minutes I get the generic error "Encrypted instances require a valid KMS key ID. (Service: Rds, Status Code: 400, Request ID: )" (RequestToken: , HandlerErrorCode: InvalidRequest)""

I've tried somethings already without success:

  • Using single region key
  • Using a Key Alias
  • Change DB instance type
  • I've created an RDS manually using the Cloudfromation generated key with success (with that I believe the problem is not the key itself)
  • Tested my RDS creation script without using the encrypted attribute (it creates successfully, with that I believe the problem is indeed within the encrypted attribute)
  • Removed managed master user password, using manual password for the user DB
  • I've tried reference the KMS Key via Ref and via GetAtt Arn

All the tentative without success.

Below a piece of my script:

"KMSKey": {
      "Type": "AWS::KMS::Key",
      "Properties": {
        "Description": "This is the KMS key used by the service customers-dev",
        "Enabled": true,
        "EnableKeyRotation": true,
        "MultiRegion": true,
        "KeyPolicy": {
          "Version": "2012-10-17",
          "Id": "customers-dev-kms-key",
          "Statement": [
            {
              "Sid": "Enable IAM User Permissions",
              "Effect": "Allow",
              "Principal": {
                "AWS": [
                  {
                    "Fn::Sub": "arn:aws:iam::MYACCOUNTID:root"
                  }
                ]
              },
              "Action": "kms:*",
              "Resource": "*"
            },
            {
              "Sid": "Allow access through RDS for all principals in the account that are authorized to use RDS",
              "Effect": "Allow",
              "Principal": {
                "AWS": "*"
              },
              "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:GenerateDataKey*",
                "kms:CreateGrant",
                "kms:ListGrants",
                "kms:DescribeKey"
              ],
              "Resource": "*",
              "Condition": {
                "StringEquals": {
                  "kms:ViaService": "rds.us-east-1.amazonaws.com",
                  "kms:CallerAccount": "MYACCOUNTID"
                }
              }
            },
            {
              "Sid": "Allow access through AWS Lambda for all principals in the account that are authorized to use AWS Lambda",
              "Effect": "Allow",
              "Principal": {
                "AWS": "*"
              },
              "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:GenerateDataKey*",
                "kms:CreateGrant",
                "kms:DescribeKey"
              ],
              "Resource": "*",
              "Condition": {
                "StringEquals": {
                  "kms:ViaService": "lambda.us-east-1.amazonaws.com",
                  "kms:CallerAccount": "MYACCOUNTID"
                }
              }
            },
            {
              "Sid": "Allow access through AWS Secrets Manager for all principals in the account that are authorized to use AWS Secrets Manager",
              "Effect": "Allow",
              "Principal": {
                "AWS": "*"
              },
              "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:CreateGrant",
                "kms:DescribeKey"
              ],
              "Resource": "*",
              "Condition": {
                "StringEquals": {
                  "kms:CallerAccount": "MYACCOUNTID",
                  "kms:ViaService": "secretsmanager.us-east-1.amazonaws.com"
                }
              }
            }
          ]
        },
        "Tags": [
          {
            "Key": "service",
            "Value": "customers"
          }
        ]
      }
    },
    "KMSKeyAlias": {
      "Type": "AWS::KMS::Alias",
      "Properties": {
        "AliasName": "alias/customers-dev-key",
        "TargetKeyId": {
          "Ref": "KMSKey"
        }
      }
    },
    "DBGroup": {
      "Type": "AWS::RDS::DBSubnetGroup",
      "Properties": {
        "DBSubnetGroupName": "customers-dev-db-group",
        "DBSubnetGroupDescription": "customers-dev DB Subnet group",
        "SubnetIds": [
          "MY SUBNETS IDS"
        ],
        "Tags": [
          {
            "Key": "service",
            "Value": "customers"
          }
        ]
      }
    },
    "RDSDatabase": {
      "DependsOn": [
        "ServerlessSecurityGroup",
        "KMSKey"
      ],
      "Type": "AWS::RDS::DBInstance",
      "Properties": {
        "MasterUsername": "MYUSER",
        "MasterUserPassword": "MYUSERPWD",
        "ManageMasterUserPassword": false,
        "StorageEncrypted": true,
        "KmsKeyId": {
          "Fn::GetAtt": [
            "KMSKey",
            "Arn"
          ]
        },
        "AllocatedStorage": 40,
        "DBInstanceIdentifier": "customers-dev-instance-db",
        "DBName": "customersdevdb",
        "DBInstanceClass": "db.t4g.small",
        "VPCSecurityGroups": [
          {
            "Fn::GetAtt": [
              "ServerlessSecurityGroup",
              "GroupId"
            ]
          }
        ],
        "DBSubnetGroupName": {
          "Ref": "DBGroup"
        },
        "Engine": "postgres",
        "AutomaticBackupReplicationRegion": "us-west-1",
        "BackupRetentionPeriod": 30,
        "CopyTagsToSnapshot": true,
        "PubliclyAccessible": false,
        "DeletionProtection": false,
        "Tags": [
          {
            "Key": "service",
            "Value": "customers"
          }
        ]
      },
      "DeletionPolicy": "Snapshot"
    }

Print showing that on console the reference works successfully but as I said after a few minutes the stack fails and rollback saying the error message about the instance needs a valid KMS Key Id.

Enter image description here

1 Answer
1
Accepted Answer

Looks like the issue happens in the destination region which you have set using AutomaticBackupReplicationRegion. You would need a KMS key in the destination region, but the CFN structure only allows setting one, which in your case is for the source region. I suggest to remove AutomaticBackupReplicationRegion and activate backup replication elsewhere, e.g. using the CLI (there you can set a destination region KMS key).

profile pictureAWS
Michael
answered a month ago
profile picture
EXPERT
reviewed a month ago
  • Yes, I've tried on a different account and saw that could be the reason then removed it and it worked, it's sad that CFN does not keep with AWS full capabilities. I was trying to avoid having to setup a structure on CFN and having separate scripts using CLI. Thanks for the quick reply.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions