SNS as message broker - Where and when to create the SNS topics?



In our application we're considering SNS (fifo) to communicate between our services, each domain should have it's own topic for other domains to subscribe to using SQS (fifo).

Due to the fact that SNS topics must be deployed in order for others to be able to subscribe to it - We ran into some deployment considerations regarding where and when to create these topics.

I hope you guys might help us get a clear answer / guidance towards the preferred approach.

Here are the options we thought could work:

  1. Each domain creates it's own topic for other domains to subscribe to Downside - If Domain B subscribes to Domain A's topic - then Domain B is dependent on Domain A's deployment, meaning that Domain B can't be deployed until Domain A is deployed (which can also yield circular dependency).
  2. Create the topic at the moment of subscription At the moment when Domain B tries to subscribe to Domain A's topic - if it doesn't exists - Domain B creates Domain A's topic. Downside - This eliminates the dependency issue from section 1, but causes a mess who's responsible for which topic
  3. Create the topics in an infra domain that should be deployed before the rest of the application is deployed This also eliminates the dependency issue Downside - now the infra domain has to know all the other domains.
  4. Create a single topic in an infra domain for the entire application and filter the messages on every subscription Downside - A single subscription has its limits (1000 subscribers ,200 filter policies).

What we're really after is the combination of section 1 (each domain creates it's own subscription) with the ability for "lazy subscription", meaning that subscribing domains won't be dependent of the topic being actually created before they can finish their deployment, something like a "pending subscription" mode.

Have we missed anything? Is it recommended to look for any AWS CDK ways (other than custom resources) to check if topics actually exists before subscribing to them?

P.S - The same applies to EventBridge and an event bus per domain.

Thanks a lot in advance :)

2 Answers

Personally I would go with option 1 but have 2 deployment steps for each domain. So Domain A Step 1 for example does all the setup that makes Domain A a service used by other domains. All the "Step 1" stuff for all domains is done before moving on to Step 2. Domain A Step 2 for example does all the setup linking Domain A as a client of other domain services.

answered a year ago
  • Dividing the deployment into 2 steps, I like it :) but I think that this might have to involve some SDK calls from the Domain A Step 2 deployment process (for example to list the topics and see if the relevant ones are available).

    Or have I missed anything? Thanks!


I would go with the 4th option: A centralized bus that everyone publishes to and subscribes on using filters. The limits you mentioned can be increased, so you can look into that.

Your Lazy doesn't seems like a good option. Decoupling the services means that the producers don't know about the consumers, or their subscriptions.

Do you really need FIFO capabilities? If you do, EventBridge is not an option for you.

profile pictureAWS
answered a year ago
  • I'll definitely look into the limits again, thanks!

    As for the lazy subscription - what I meant was not for the producers to know the consumers but it could ease things for us if for example the topic subscription could've been using a dynamic arn.

    We need the FIFO capabilities, mainly because of the deduplication and the ability to group messages by an id.

    Thanks Uri.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions