Subscribing to SNS topic in another account/region not working for SQS queue via CloudFormation yaml template, but working through AWS console.
Hello,
My team has a cloud formation YAML template includes a map of SNS topic ARNs owned by a different team's account that our SQS queue subscribes to, the queue policy that permits SNS to send message, and the wiring for SQS to listen to SNS subscription. This configuration has been working for multiple years in production. Recently we made the change in the map for our SQS queue in us-west-2 to switch to subscribing to their topic ARN in ap-southeast-1. Upon deploying the change, we see the error:
Resource handler returned message: "Invalid parameter: TopicArn (Service: Sns, Status Code: 400, Request ID: bf308ba0-b205-5ae8-b431-eab664307eae) (SDK Attempt Count: 1)" (RequestToken: 0bb14da9-a7dc-89c7-348c-69b05886f1ae, HandlerErrorCode: InvalidRequest)
We verified with the other team that our team's account id has been allowlisted in the access policy for the SNS topic and have verified that the topic definitely exists by going into the AWS console and manually going to our queue and pasting the TopicArn from our CF template into the console and successfully pressing Subscribe.
If it works via the AWS console, why is it not working from the CF template?
Steps we have taken to investigate:
- When deploying changes to stack, I looked at Cloud Formation change set and verified the TopicArn in the template is correct.
- Language
- English
- Newest
- Most votes
- Most comments
The issue you're experiencing is related to cross-account and cross-region SNS to SQS subscription permissions when using CloudFormation versus the console.
When you subscribe to an SNS topic in another account via the AWS console, the console handles some of the permission requirements behind the scenes. However, when using CloudFormation, you need to ensure all the necessary permissions are explicitly configured.
For cross-account SNS to SQS subscriptions to work properly, two sets of permissions are required:
- The SNS topic owner (the other team's account) must modify their topic's access policy to allow your account to subscribe. This policy should include a statement like:
{
"Effect": "Allow",
"Principal": { "AWS": "arn:aws:iam:[YOUR-ACCOUNT-ID]:root" },
"Action": "sns:Subscribe",
"Resource": "arn:aws:sns:ap-southeast-1:[THEIR-ACCOUNT-ID]:TopicName"
}
- Your SQS queue's access policy must allow the SNS topic to send messages to it.
The cross-region aspect adds another layer of complexity. Make sure that:
- The SNS topic's access policy allows cross-region subscriptions
- Your queue's access policy permits the SNS topic to send messages across regions
Since it works via the console but not via CloudFormation, it's likely that the CloudFormation service role doesn't have the necessary permissions to create the subscription on your behalf, or there might be a specific formatting issue with how the cross-region ARN is specified in your template.
I recommend checking that your CloudFormation template includes both the subscription resource and the proper queue policy that allows the specific SNS topic ARN from the other region to publish to your queue.
Sources
Subscribing a queue to an Amazon SNS topic using the Amazon SQS console - Amazon Simple Queue Service
Sending Amazon SNS messages to an Amazon SQS queue in a different account - Amazon Simple Notification Service
Enforce resource configuration to control access to new features with AWS | AWS Security Blog
Thank you for the response, I've verified that the SNS topic owner has modified their topic's access policy (especially as subscribing works from the console). From my own template, our team has a policy like the below. Does any of it indicate that it would not work for a cross region use case?
SQS Queue permitting SNS to send message
RefreshSpiceEventQueuePolicy: Type: AWS::SQS::QueuePolicy Properties: PolicyDocument: Version: '2012-10-17' Statement: - Sid: allow-sns-publish-access-to-sqs Effect: Allow Principal: AWS: '' Action: - sqs:SendMessage Resource: '' Condition: ArnEquals: aws:SourceArn: Fn::FindInMap: - S3EventSNSTopicARNMap - Ref: 'AWS::Region' - Fn::FindInMap: ['StageMapping', {Ref: 'Stage'}, 'value'] Queues: - Ref: EventQueue