EventBridge DeadLetterConfig Drift False Positive

0

I have the following CloudFormation template that notifies users of AWS Health events via an SNS topic:

AWSTemplateFormatVersion: "2010-09-09"
Description: Subscribe to account-specific and global events from AWS Health
Parameters:
  Email:
    Type: String
    Description: The email address that will receive SNS notifications for the health alerts
    # Simple regex from https://stackoverflow.com/a/201378
    AllowedPattern: "(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*|\"(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21\\x23-\\x5b\\x5d-\\x7f]|\\\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])*\")@(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\\[(?:(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9]))\\.){3}(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9])|[a-z0-9-]*[a-z0-9]:(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21-\\x5a\\x53-\\x7f]|\\\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])+)\\])"
Resources:
  EventBridgeRule:
    Type: AWS::Events::Rule
    Properties:
      Name: USEast1HealthRule
      Description: Push AWS Health notifications to an SNS topic
      EventPattern:
        source:
          - aws.health
        detail-type:
          - AWS Health Event
        detail:
          service:
            # Services to get notified for health alerts
            - CLOUDFORMATION
            - S3
            - CLOUDFRONT
            - APIGATEWAY
            - LAMBDA
            - DYNAMODB
            - IAM
            - CODEBUILD
            - CODEDEPLOY
            - CODEPIPELINE
            - KMS
            - SQS
            - ECR
            - COGNITO
            - CLOUDWATCH
            - XRAY
      State: ENABLED
      Targets:
        - Id: EmailTarget
          Arn: !Ref SNSTopic
          # Failed events are retried up to 185 times within 24 hours by default
          DeadLetterConfig:
            Arn: !GetAtt DLQ.Arn
  SNSTopic:
    Type: AWS::SNS::Topic
    Properties:
      # Encrypt using the default SNS SSE key
      # Key aliases: aws kms list-aliases
      KmsMasterKeyId: alias/aws/sns
      Subscription:
        - Endpoint: !Ref Email
          Protocol: email
      TopicName: AWSHealthNotifications
  SNSTopicPolicy:
    Type: AWS::SNS::TopicPolicy
    Properties:
      PolicyDocument:
        Statement:
          # Allow EventBridge to send email alerts using SNS
          - Sid: AllowEventBridgeServicePrincipalWriteOnly
            Effect: Allow
            Principal:
              Service: events.amazonaws.com
            Action: sns:Publish
            Resource: !Ref SNSTopic
            Condition:
              ArnEquals:
                "aws:SourceArn": !GetAtt EventBridgeRule.Arn
          - Sid: AllowSSLRequestsOnly
            Effect: Deny
            Principal: "*"
            Action: sns:Publish
            Resource: !Ref SNSTopic
            Condition:
              Bool:
                "aws:SecureTransport": false
      Topics:
        - !Ref SNSTopic
  DLQ:
    Type: AWS::SQS::Queue
    DeletionPolicy: Delete
    UpdateReplacePolicy: Delete
    Properties:
      # Defaults:
      # DelaySeconds (delay queue) = 0s (time before messages appear in the queue)
      # MessageRetentionPeriod = 4 days (345,600s) (time messages stays in the queue)
      # VisibilityTimeout = 30s (time before messages reappear in the queue)
      MessageRetentionPeriod: 345600
      ReceiveMessageWaitTimeSeconds: 5 # long polling if > 0
      SqsManagedSseEnabled: true
  DLQPolicy:
    Type: AWS::SQS::QueuePolicy
    Properties:
      # Allow EventBridge to send messages to the DLQ
      PolicyDocument:
        Statement:
          - Sid: AllowEventBridgeServicePrincipalWriteOnly
            Effect: Allow
            Principal:
              Service: events.amazonaws.com
            Action: sqs:SendMessage
            Resource: !GetAtt DLQ.Arn
            Condition:
              ArnEquals:
                "aws:SourceArn": !GetAtt EventBridgeRule.Arn
          - Sid: AllowSSLRequestsOnly
            Effect: Deny
            Principal: "*"
            Action: "sqs:*"
            Resource: !GetAtt DLQ.Arn
            Condition:
              Bool:
                "aws:SecureTransport": false
      Queues:
        - !Ref DLQ
Outputs:
  DLQURL:
    Description: DLQ URL for EventBridge
    Value: !Ref DLQ

When I detect drift on this stack, I get the following output:

{
    "StackResourceDrifts": [
        {
            "StackId": "arn:aws:cloudformation:us-east-1:***:stack/Health-Alert-Stack/f035cfc0-162a-11ee-a14e-12f8f4f0e2d3",
            "LogicalResourceId": "EventBridgeRule",
            "PhysicalResourceId": "USEast1HealthRule",
            "ResourceType": "AWS::Events::Rule",
            "ExpectedProperties": "{\"Description\":\"Push AWS Health notifications to an SNS topic\",\"EventPattern\":\"{\\\"detail-type\\\":[\\\"AWS Health Event\\\"],\\\"source\\\":[\\\"aws.health\\\"],\\\"detail\\\":{\\\"service\\\":[\\\"CLOUDFORMATION\\\",\\\"S3\\\",\\\"CLOUDFRONT\\\",\\\"APIGATEWAY\\\",\\\"LAMBDA\\\",\\\"DYNAMODB\\\",\\\"IAM\\\",\\\"CODEBUILD\\\",\\\"CODEDEPLOY\\\",\\\"CODEPIPELINE\\\",\\\"KMS\\\",\\\"SQS\\\",\\\"ECR\\\",\\\"COGNITO\\\",\\\"CLOUDWATCH\\\",\\\"XRAY\\\"]}}\",\"Name\":\"USEast1HealthRule\",\"State\":\"ENABLED\",\"Targets\":[{\"Arn\":\"arn:aws:sns:us-east-1:***:AWSHealthNotifications\",\"DeadLetterConfig\":{\"Arn\":\"arn:aws:sqs:us-east-1:***:Health-Alert-Stack-DLQ-8Q2i6BSM97kI\"},\"Id\":\"EmailTarget\"}]}",
            "ActualProperties": "{\"Description\":\"Push AWS Health notifications to an SNS topic\",\"EventPattern\":\"{\\\"detail-type\\\":[\\\"AWS Health Event\\\"],\\\"source\\\":[\\\"aws.health\\\"],\\\"detail\\\":{\\\"service\\\":[\\\"CLOUDFORMATION\\\",\\\"S3\\\",\\\"CLOUDFRONT\\\",\\\"APIGATEWAY\\\",\\\"LAMBDA\\\",\\\"DYNAMODB\\\",\\\"IAM\\\",\\\"CODEBUILD\\\",\\\"CODEDEPLOY\\\",\\\"CODEPIPELINE\\\",\\\"KMS\\\",\\\"SQS\\\",\\\"ECR\\\",\\\"COGNITO\\\",\\\"CLOUDWATCH\\\",\\\"XRAY\\\"]}}\",\"Name\":\"USEast1HealthRule\",\"State\":\"ENABLED\",\"Targets\":[{\"Arn\":\"arn:aws:sns:us-east-1:***:AWSHealthNotifications\",\"Id\":\"EmailTarget\"}]}",
            "PropertyDifferences": [
                {
                    "PropertyPath": "/Targets/0/DeadLetterConfig",
                    "ExpectedValue": "{\"Arn\":\"arn:aws:sqs:us-east-1:***:Health-Alert-Stack-DLQ-8Q2i6BSM97kI\"}",
                    "ActualValue": "null",
                    "DifferenceType": "REMOVE"
                }
            ],
            "StackResourceDriftStatus": "MODIFIED",
            "Timestamp": "2023-08-01T02:24:13.184000+00:00"
        }
    ]
}

CloudFormation is detecting drift on the DeadLetterConfig of the EventBridge target, but I see in the management console and the CLI that the DLQ is configured:

aws events list-targets-by-rule --rule USEast1HealthRule:

{
    "Targets": [
        {
            "Id": "EmailTarget",
            "Arn": "arn:aws:sns:us-east-1:***:AWSHealthNotifications",
            "DeadLetterConfig": {
                "Arn": "arn:aws:sqs:us-east-1:***:Health-Alert-Stack-DLQ-8Q2i6BSM97kI"
            }
        }
    ]
}

I tried the following to no avail, making me think this is a false positive from CloudFormation:

  • I waited a few days to see if the drift error would go away, but it didn't.
  • I'm running the drift analysis as an admin user, so there shouldn't be any access issues.
  • I double-checked that everything was deployed in the us-east-1 region.
  • I tried commenting and uncommenting the DeadLetterConfig block to "delete" the resource, but the drift error remained.
  • I deployed the same stack to a sandbox account and created a similar stack for Trusted Advisor events, and I'm getting the same drift error.
1 Antwort
0
Akzeptierte Antwort

Hi, yes, it seems to be an issue with improper drift detection. The best is to open a support case via the AWS console: Service Teams usually do not monitor re:Post for such issue. You will get proper follow-up on your problem via the Support case.

In the meantime, you may want to apply the CFN protection for "false drift detection": see https://docs.aws.amazon.com/cloudformation-cli/latest/userguide/resource-type-model-false-drift.html

You can use this mechanism to at least hide the current drift issue until it gets fixed.

Best, Didier

profile pictureAWS
EXPERTE
beantwortet vor 10 Monaten
  • Ok, thank you. I thought to post here since the docs mentioned how to report false positives:

    If you encounter any false positive, send us your comments using the feedback link in the CloudFormation console, or reach out to us through the AWS forums at https://forums.aws.amazon.com/.

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen