- 最新
- 最多得票
- 最多評論
You can't do that. The messages will be distributed across all consumers without a way to filter the messages.
You could use EventBridge and create multiple rules, with different filters that will distribute to different queues.
Hi, if you have a single queue, all readers can read from this queue wihtout acknowledging in a non-destructive manner. Only the reader interested in it will delete the message after he successfully got it.
This scenario implies useless reads by all readers except 1. So, these additional API calls have to be accounted for. Of course, 1queue per reader would be better from API call perspective. It all boils down to how many of such queues you need.
Amazon SQS doesn't automatically delete a message after retrieving it
for you, in case you don't successfully receive the message (for example,
if the consumers fail or you lose connectivity). To delete a message, you
must send a separate request which acknowledges that you've successfully received
and processed the message. Note that you must receive a message before you can
delete it.
What is the meaning of a non-destructive manner in this case?
When a consumer reads a message from AWS SQS without acknowledging it, will the other consumers also be able to read the same message? Wouldn't that message become invisible to all other consumers while one consumer is already reading it?
I don't believe SQS is the right service for this use case. You can consider following alternatives :
- Amazon SNS that allows message filtering at the subscriber but it might or might not fit your use case as it is based on a pub/sub methodology and not a Producer/Consumer
- Amazon MSK : you can benefit from full power of Kafka to create topics and direct each consumer to consume from its topic. You can deploy Amazon MSK as well in a serverless manner as well as cluster manner.
- You can drive a serverless consumer using step-function that receives all the messages and triggers the right consumer based on the message categorization (this means that the consumers are better to be Lambda based). In this way you are pushing the consumption logic to the step function and not SQS.
To me it seems that Kinesis Data Streams are a more applicable solution here. It allows for multiple readers/consumers for each message; each of which can decide whether to process the message or ignore it.
I'd note that the two services (Kinesis and SQS) have different limits in terms of data and message volumes but it might be worth considering.
相關內容
- 已提問 7 個月前
- AWS 官方已更新 1 年前
- AWS 官方已更新 2 年前
- AWS 官方已更新 3 年前
- AWS 官方已更新 7 個月前
Multiple queues are not considered as an option in the proposed architecture. In the current architecture, it is necessary to have a single AWS SQS that is consumed by several consumers, more than 100 in most cases.