Cloudformation failing to create RDS encrypted database with created KMS Key "Encrypted instances require a valid KMS key ID"

0

I'm creating my structure using CloudFormation but I'm stuck with an error that doesn't make sense. To summarize, I'm creating a KMS Key and using it in the Stack itself to configure a Postgres Encrypted RDS.

The key is always created successfully, in the database I use DependsOn to make sure that Cloudformation is following the sequence.

The process runs successfully until the last step where it waits for an ok to create the RDS, I see the instance successfully in the console with the key created correctly referenced, but after a few minutes I get the generic error "Encrypted instances require a valid KMS key ID. (Service: Rds, Status Code: 400, Request ID: )" (RequestToken: , HandlerErrorCode: InvalidRequest)""

I've tried somethings already without success:

  • Using single region key
  • Using a Key Alias
  • Change DB instance type
  • I've created an RDS manually using the Cloudfromation generated key with success (with that I believe the problem is not the key itself)
  • Tested my RDS creation script without using the encrypted attribute (it creates successfully, with that I believe the problem is indeed within the encrypted attribute)
  • Removed managed master user password, using manual password for the user DB
  • I've tried reference the KMS Key via Ref and via GetAtt Arn

All the tentative without success.

Below a piece of my script:

"KMSKey": {
      "Type": "AWS::KMS::Key",
      "Properties": {
        "Description": "This is the KMS key used by the service customers-dev",
        "Enabled": true,
        "EnableKeyRotation": true,
        "MultiRegion": true,
        "KeyPolicy": {
          "Version": "2012-10-17",
          "Id": "customers-dev-kms-key",
          "Statement": [
            {
              "Sid": "Enable IAM User Permissions",
              "Effect": "Allow",
              "Principal": {
                "AWS": [
                  {
                    "Fn::Sub": "arn:aws:iam::MYACCOUNTID:root"
                  }
                ]
              },
              "Action": "kms:*",
              "Resource": "*"
            },
            {
              "Sid": "Allow access through RDS for all principals in the account that are authorized to use RDS",
              "Effect": "Allow",
              "Principal": {
                "AWS": "*"
              },
              "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:GenerateDataKey*",
                "kms:CreateGrant",
                "kms:ListGrants",
                "kms:DescribeKey"
              ],
              "Resource": "*",
              "Condition": {
                "StringEquals": {
                  "kms:ViaService": "rds.us-east-1.amazonaws.com",
                  "kms:CallerAccount": "MYACCOUNTID"
                }
              }
            },
            {
              "Sid": "Allow access through AWS Lambda for all principals in the account that are authorized to use AWS Lambda",
              "Effect": "Allow",
              "Principal": {
                "AWS": "*"
              },
              "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:GenerateDataKey*",
                "kms:CreateGrant",
                "kms:DescribeKey"
              ],
              "Resource": "*",
              "Condition": {
                "StringEquals": {
                  "kms:ViaService": "lambda.us-east-1.amazonaws.com",
                  "kms:CallerAccount": "MYACCOUNTID"
                }
              }
            },
            {
              "Sid": "Allow access through AWS Secrets Manager for all principals in the account that are authorized to use AWS Secrets Manager",
              "Effect": "Allow",
              "Principal": {
                "AWS": "*"
              },
              "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:CreateGrant",
                "kms:DescribeKey"
              ],
              "Resource": "*",
              "Condition": {
                "StringEquals": {
                  "kms:CallerAccount": "MYACCOUNTID",
                  "kms:ViaService": "secretsmanager.us-east-1.amazonaws.com"
                }
              }
            }
          ]
        },
        "Tags": [
          {
            "Key": "service",
            "Value": "customers"
          }
        ]
      }
    },
    "KMSKeyAlias": {
      "Type": "AWS::KMS::Alias",
      "Properties": {
        "AliasName": "alias/customers-dev-key",
        "TargetKeyId": {
          "Ref": "KMSKey"
        }
      }
    },
    "DBGroup": {
      "Type": "AWS::RDS::DBSubnetGroup",
      "Properties": {
        "DBSubnetGroupName": "customers-dev-db-group",
        "DBSubnetGroupDescription": "customers-dev DB Subnet group",
        "SubnetIds": [
          "MY SUBNETS IDS"
        ],
        "Tags": [
          {
            "Key": "service",
            "Value": "customers"
          }
        ]
      }
    },
    "RDSDatabase": {
      "DependsOn": [
        "ServerlessSecurityGroup",
        "KMSKey"
      ],
      "Type": "AWS::RDS::DBInstance",
      "Properties": {
        "MasterUsername": "MYUSER",
        "MasterUserPassword": "MYUSERPWD",
        "ManageMasterUserPassword": false,
        "StorageEncrypted": true,
        "KmsKeyId": {
          "Fn::GetAtt": [
            "KMSKey",
            "Arn"
          ]
        },
        "AllocatedStorage": 40,
        "DBInstanceIdentifier": "customers-dev-instance-db",
        "DBName": "customersdevdb",
        "DBInstanceClass": "db.t4g.small",
        "VPCSecurityGroups": [
          {
            "Fn::GetAtt": [
              "ServerlessSecurityGroup",
              "GroupId"
            ]
          }
        ],
        "DBSubnetGroupName": {
          "Ref": "DBGroup"
        },
        "Engine": "postgres",
        "AutomaticBackupReplicationRegion": "us-west-1",
        "BackupRetentionPeriod": 30,
        "CopyTagsToSnapshot": true,
        "PubliclyAccessible": false,
        "DeletionProtection": false,
        "Tags": [
          {
            "Key": "service",
            "Value": "customers"
          }
        ]
      },
      "DeletionPolicy": "Snapshot"
    }

Print showing that on console the reference works successfully but as I said after a few minutes the stack fails and rollback saying the error message about the instance needs a valid KMS Key Id.

Enter image description here

1 Antwort
1
Akzeptierte Antwort

Looks like the issue happens in the destination region which you have set using AutomaticBackupReplicationRegion. You would need a KMS key in the destination region, but the CFN structure only allows setting one, which in your case is for the source region. I suggest to remove AutomaticBackupReplicationRegion and activate backup replication elsewhere, e.g. using the CLI (there you can set a destination region KMS key).

profile pictureAWS
Michael
beantwortet vor einem Monat
profile picture
EXPERTE
überprüft vor einem Monat
  • Yes, I've tried on a different account and saw that could be the reason then removed it and it worked, it's sad that CFN does not keep with AWS full capabilities. I was trying to avoid having to setup a structure on CFN and having separate scripts using CLI. Thanks for the quick reply.

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen