By using AWS re:Post, you agree to the AWS re:Post Terms of Use

Clarification about SNS Fifo max throughput and quotas

0

Hi,

I would like to clarify some doubts about the SNS Fifo.

About the quota of 3,000 messages per second:

  • Link 2 says it is by topic.
  • Link 3 is a bit ambiguous, in Statement 1 it suggests it is by region, in Statement 2 it suggests it is by topic.
  • Link 4, although it is not official documentation, was validated by someone considered an expert, and says it is by region.

My understanding is as follows: A single SNS FIFO topic has a default quota of 3,000 messages per second per topic. This quota is independent of the message groups, it is validated on the topic as a whole considering all messages from all message groups. In addition, there is a maximum throughput of 300 messages per second per topic/message group. Therefore, it is possible to have 300 messages per second in 10 different message groups, totaling 3,000 messages per second, without any throttling. Is this correct?

I understand that if one of these 2 limits is exceeded, AWS will throttle, generating a 429. Is this correct?

1 Answer
0

Your understanding is mostly correct, but there are some important clarifications to make:

  1. SNS FIFO topic throughput: The current quota for Amazon SNS FIFO topics is indeed 3,000 messages per second per topic. This is a significant increase from the previous limit and applies to each individual FIFO topic, not per region.

  2. Message group throughput: Within a FIFO topic, each individual message group can deliver a maximum of 300 messages per second. This is correct and helps maintain strict ordering within each message group.

  3. Topic vs. Region quota: The quota of 3,000 messages per second is applied per topic, not per region. This means you can have multiple FIFO topics in a region, each capable of handling up to 3,000 messages per second.

  4. Maximizing throughput: Your understanding about being able to have 300 messages per second in 10 different message groups, totaling 3,000 messages per second without throttling, is correct. This is an optimal way to utilize the full capacity of a FIFO topic while maintaining message ordering within each group.

  5. Throttling: If either the per-topic limit (3,000 messages/second) or the per-message group limit (300 messages/second) is exceeded, AWS will indeed throttle the requests, likely resulting in a 429 (Too Many Requests) error.

It's important to note that to achieve maximum throughput, you should distribute your messages evenly over a large number of message group IDs. This allows SNS to process messages from different groups in parallel while maintaining strict ordering within each group.

Also, keep in mind that when working with high-throughput FIFO topics, it's recommended to enable high throughput mode on any subscribed Amazon SQS FIFO queues to ensure they can handle the increased message flow.
Sources
Amazon SNS increases default FIFO topic throughput by 10x to 3,000 messages per second
Amazon SNS message grouping for FIFO topics - Amazon Simple Notification Service

profile picture
answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions