MSK with SASL/SCRAM authentication. I can access the cluster but cant perform any action



We're currently building an MSK cluster. We use 2 types of authentication for 2 different clients.

The IAM authentication works fine.

But for the SASL/SCRAM authentication that it's not the case, we created a secret for username/password connection and linked it to the MSK cluster.

Using our client or with a UI client for windows it's the same issue we have a connection but with limited functionalities (like creating a topic or reading one).

Please take a look at the error the UI clients returns :

Could not complete DescribeConfigs action: you can try to continue with limited functionality. ClusterAuthorizationException: Cluster authorization failed. Make sure that your user has all access rights (DescribeConsumerGroups, DescribeCluster, DescribeConfigs) for full functionality.

Also take a look at our AWS secret policy JSON file :

  "Version" : "2012-10-17",
  "Statement" : [ {
    "Sid" : "......",
    "Effect" : "Allow",
    "Principal" : {
      "Service" : ""
    "Action" : "secretsmanager:getSecretValue",
    "Resource" : "arn:aws:secretsmanager:eu-west-3...............Z"
  } ]

Do we need to modify the ACLs policies directly on the Kafka instance ? How?


1 Answer

It seems like you've changed to false.

If that's the case, then you won't have access with SCRAM auth method until you enable ACLs. So, you need to run kafka-acls command and add permissions to the user you use to read topics, or write...

For example, you will need to use a user that already has permissions to set up ACLs, alternatively, you can use unauthenticated method, or zookeeper instead of bootstrap servers, so authentication is not checked:

kafka/bin/ --bootstrap-server msk:9096 \
  --command-config adminclient-configs.conf \
  --add \
  --allow-principal User:boris \
  --allow-principal User:ed \
  --operation read \
  --operation write \
  --topic my-topic
answered a month ago
profile picture
reviewed a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions