How do I fix the circular dependency between an AWS Lambda permission and target group resources in AWS CloudFormation?

Short description

You get a circular dependency if you provide AWS::Lambda::Permission to AWS::ElasticLoadBalancingV2::TargetGroup instead of LoadBalancer to restrict access to your Lambda function. This is because the Lambda permission must exist before you associate the Lambda function to the target group that you want to create. When you create the Lambda permission, you must associate the permission with an AWS Principal and SourceARN (in this case, a target group). That's why the target group must exist before creating the Lambda permission. To fix this circular dependency, you can use a Lambda-backed custom resource.

Resolution

1.    Create a file named index.py that includes the following:

Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

Permission is hereby granted, free of charge, to any person obtaining a copy of this
software and associated documentation files (the "Software"), to deal in the Software
without restriction, including without limitation the rights to use, copy, modify,
merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

def handler(event, context): 
    return {
      "statusCode": 200,
      "statusDescription": "HTTP OK",
      "isBase64Encoded": False,
      "headers": {
         "Content-Type": "text/html"
      },
      "body": "<h1>Hello from Lambda!</h1>"
    }

Note: The AWS Lambda function in index.py displays an HTML page with the message "Hello from Lambda!" when the function is invoked.

2.    Add the index.py file to an archive file named website.zip.

3.    To create a .zip file of index.py, run the following command:

$zip website.zip index.py

4.    Create a file called cfnresponse.py based on the AWS CloudFormation template on AWS GitHub.

5.    Create a filled called custom.py based on the following:

Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

Permission is hereby granted, free of charge, to any person obtaining a copy of this
software and associated documentation files (the "Software"), to deal in the Software
without restriction, including without limitation the rights to use, copy, modify,
merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

import logging
import boto3, json, botocore
import cfnresponse
#Define logging properties for 'logging'
log = logging.getLogger()
log.setLevel(logging.INFO)
#Main Lambda function to be executed
def lambda_handler(event, context):
    try:
        if event['RequestType'] == "Create" or event['RequestType'] == "Update":
            # print the event type sent from cloudformation
            log.info ("'" + str(event['RequestType']) + "' event was sent from CFN")
            # Imput Parameters
            TargetGroupARN = event['ResourceProperties']['TargetGroupARN']
            LambdaFunctionARN = event['ResourceProperties']['LambdaFunctionARN']
            log.info("TargetGroup ARN value is:" + TargetGroupARN)
            log.info("Lambda Function ARN value is:" + LambdaFunctionARN)
            responseData = {}
            # ELBV2 initilize
            client = boto3.client('elbv2')
            # Initilize Vars
            response = ''
            error_msg = ''
            # Make the 1st API call to get the Lambda policy and extract SID of the initial permissions that were created by the CFN template.
            try:
                response = client.register_targets(
                    TargetGroupArn=TargetGroupARN,
                    Targets=[
                        {
                            'Id': LambdaFunctionARN
                        },
                    ]
                )
            except botocore.exceptions.ClientError as e:
                error_msg = str(e)
            if error_msg:
                log.info("Error Occured:" + error_msg)
                response_msg = error_msg
                # TODO: SIGNAL BACK TO CLOUDFORMATION
                log.info("Trying to signal FAILURE back to cloudformation.")
                responseData = {"Message" : response_msg, "Function" : context.log_stream_name}
                cfnresponse.send(event, context, cfnresponse.FAILED, responseData)
            else:
                response_msg = "Successfully Added Lambda(" +  LambdaFunctionARN + ") as Target"
                log.info(response_msg)
                # TODO: SIGNAL BACK TO CLOUDFORMATION
                log.info("Trying to signal SUCCESS back to cloudformation.")
                responseData = {"Message" : response_msg, "Function" : context.log_stream_name}
                cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData)
            
            # log the end of the create event
            log.info ("End of '" + str(event['RequestType']) + "' Event")
        elif "Delete" in str(event['RequestType']):
            # print the event type sent from cloudformation
            log.info ("'Delete' event was sent from CFN")
            
            # TODO: DELETE THINGS
            log.info("TODO: DELETE THINGS")
            
            # TODO: SIGNAL BACK TO CLOUDFORMATION
            log.info("Trying to signal SUCCESS back to cloudformation.")
            responseData = {"Function" : context.log_stream_name}
            cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData)
            
            # log the end of the Delete event
            log.info ("End of 'Delete' Event")
        else:
            log.info ("RequestType sent from cloudformation is invalid.")
            log.info ("Was expecting 'Create', 'Update' or 'Delete' RequestType(s).")
            log.info ("The detected RequestType is : '" + str(event['RequestType']) + "'")
            
            #TODO: SIGNAL BACK TO CLOUDFORMATION
            log.info("Trying to signal FAILURE back to cloudformation due to invalid request type.")
            responseData={"Function" : context.log_stream_name}
            cfnresponse.send(event, context, cfnresponse.FAILED, responseData)
            
            
    except Exception as e:
        log.info ("Function failed due to the following error:")
        print (e)
        
        #TODO: SIGNAL BACK TO CLOUDFORMATION
        log.info("Trying to signal FAILURE back to cloudformation due to the error.")
        responseData={"Function" : context.log_stream_name}
        cfnresponse.send(event, context, cfnresponse.FAILED, responseData)

Note: The Lambda function in custom.py performs the RegisterTargets API call and signals AWS CloudFormation on completion of that API call.

6.    Add the cfnresponse.py and custom.py files to an archive file named custom.zip. For example:

zip custom.zip cfnresponse.py custom.py

Note: The custom.py file uses the Boto3 API call RegisterTargets. This API call registers the specified targets with the specified target group. For more information, see the register_targets Python snippet from the Boto3 website. The cfnresponse.py file contains a function that pushes logs and notifications to Amazon CloudWatch and AWS CloudFormation.

7.    Create an AWS CloudFormation template using the following YAML template:

Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

Permission is hereby granted, free of charge, to any person obtaining a copy of this
software and associated documentation files (the "Software"), to deal in the Software
without restriction, including without limitation the rights to use, copy, modify,
merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

AWSTemplateFormatVersion: 2010-09-09
Description: HelloWorld Lambda function template for Application Load Balancer Lambda as target
Parameters:
  # VPC in which the LoadBalancer and the LoadBalancer SecurityGroup will be created
  VpcId:
      Type: AWS::EC2::VPC::Id
  # Subnets in which the LoadBalancer will be created.
  Subnets:
    Type: List<AWS::EC2::Subnet::Id>
  # Name of the TargetGroup
  TargetGroupName: 
    Type: String
    Default: 'MyTargets'
  # Name of the S3 bucket where custom.zip and website.zip are uploaded to 
  S3BucketName:
    Type: String
    Default: you-s3-bucket-name
Resources:
  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
          Action:
          - sts:AssumeRole
      Path: "/"
      Policies:
      - PolicyName: root
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
            - '*'
            Resource: '*'
  # Lambda function which displays an HTML page with "Hello from Lambda!" message upon invocation
  HelloWorldFunction1234:
    Type: 'AWS::Lambda::Function'
    Properties:
      Code:
        S3Bucket: !Ref S3BucketName
        S3Key: website.zip
      FunctionName: testLambda
      MemorySize: 128
      Handler: index.handler
      Timeout: 30
      Runtime: python3.7
      Role: !GetAtt LambdaExecutionRole.Arn
  LoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Scheme: internet-facing 
      Subnets: !Ref Subnets
      SecurityGroups:
      - !Ref LoadBalancerSecurityGroup
  HttpListener:
    Type: 'AWS::ElasticLoadBalancingV2::Listener'
    Properties:
      DefaultActions:
      - TargetGroupArn: !Ref TargetGroup
        Type: forward
      LoadBalancerArn: !Ref LoadBalancer
      Port: 80
      Protocol: HTTP
  LoadBalancerSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow http to client host
      VpcId: !Ref VpcId
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: 80
        ToPort: 80
        CidrIp: 0.0.0.0/0
  HelloWorldFunctionInvokePermission:
    Type: AWS::Lambda::Permission
    Properties:
      FunctionName: !GetAtt HelloWorldFunction1234.Arn
      Action: 'lambda:InvokeFunction'
      Principal: elasticloadbalancing.amazonaws.com
      SourceArn: !Sub 
        - >-
          arn:aws:elasticloadbalancing:${AWS::Region}:${AWS::AccountId}:${TGFullName}
        - TGFullName: !GetAtt TargetGroup.TargetGroupFullName
  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: !Ref TargetGroupName
      TargetType: lambda
  # Custom resource for Lambda function - "HelloWorldFunction1234"
  HelloWorld:
    DependsOn: [TargetGroup, HelloWorldFunctionInvokePermission]
    Type: Custom::HelloWorld
    Properties:
      ServiceToken: !GetAtt TestFunction.Arn
      # Input parameters for the Lambda function
      LambdaFunctionARN: !GetAtt HelloWorldFunction1234.Arn
      TargetGroupARN: !Sub 
        - >-
          arn:aws:elasticloadbalancing:${AWS::Region}:${AWS::AccountId}:${TGFullName}
        - TGFullName: !GetAtt TargetGroup.TargetGroupFullName
  # Lambda function that performs RegisterTargets API call
  TestFunction:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        S3Bucket: !Ref S3BucketName
        S3Key: custom.zip
      Handler: custom.lambda_handler
      Role: !GetAtt LambdaExecutionRole.Arn
      Runtime: python3.7
      Timeout: '5'
Outputs:
  Message:
    Description: Message returned from Lambda
    Value: !GetAtt 
      - HelloWorld
      - Message

8.    Pass in your values for the S3BucketName, Subnets, VpcId, and TargetGroupName parameters to the AWS CloudFormation stack that you created using the template in step 7.

9.    Create the stack.

10.    After all the API calls are completed, go to the Outputs section of the AWS CloudFormation console, and then look for the following message:

Successfully Added Lambda(arn:aws:lambda:us-east-1:123456789:function:testLambda) as Target

---