Using MSK as trigger to a Lambda with SASL/SCRAM Authentication

0

Hi, I have set up a MSK cluster with SASL/SCRAM authentication. I have stored the username and password in a secret using AWS Secrets Manager. Now I am trying to set the topic in the MSK cluster as an event source to a Lambda function. In order to do so, I am following this documentation: https://aws.amazon.com/blogs/compute/using-amazon-msk-as-an-event-source-for-aws-lambda/ However the above documentation is for unauthenticated protocol. So I tried to add the authentication and the secret. I also added a policy in the execution role of the Lambda function that lets it read the secret value:

"Version": "2012-10-17",
"Statement": [
    {
        "Effect": "Allow",
        "Action": [
            "secretsmanager:*"
        ],
        "Resource": [
            "arn:aws:secretsmanager:****:*******:secret:*"
        ]
    },
    {
        "Effect": "Allow",
        "Action": "secretsmanager:ListSecrets",
        "Resource": "*"
    }
]}

When I am trying to add the trigger, I am getting the error: An error occurred when creating the trigger:

Cannot access secret manager value arn:aws:secretsmanager:::secret:*******. Please ensure the role can perform the 'secretsmanager:GetSecretValue' action on your broker in IAM. (Service: AWSLambda; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: ****; Proxy: null)

I am not able to understand this error since I have included in the policy all the Actions from "secretsmanager" on all the Resources in my account. Can someone help?

2 Answers
0

Hi, Thanks for your reply. Since I would like the MSK Cluster have public access, I cannot have clusters hosted in private subnets inside the VPC. I do have a NAT Gateway running in the Public subnet. I am able to produce and consume messages to the topic in this cluster from terminal, but when trying to redirect the message to the Lambda function as a payload, I am facing the above mentioned problem.

answered 2 years ago
  • Are you deploying the Lambda function with VPC access? Is that the same VPC where the MSK cluster is running?

0

Since Lambda must have access to your VPC to use Amazon MSK as a trigger, you need to make sure that, if the subnets you chose are private, there is connectivity to a NAT Gateway or to a Secrets Manager VPC Endpoint to access Secrets Manager.

profile pictureAWS
EXPERT
Tasio
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions