SQS FIFO Queue w/ every message having its own unique message group ID is long polling for 0 when queue has more than 10k items

0

I have a FIFO queue setup where each single message sent is sent with a guaranteed unique message group ID.

DeduplicationScope = perMessageGroup FifoThroughputLimit=perMessageGroupId

I've been using this queue fine for months until today when something very strange happened.

At one point I had about 700 messages in flight and around 15k messages visible in queue. At this point when I made a recieveMessage request with with long polling with a message return count of 10, i was getting zero messages returned consistently for like hours. Then randomly all of a sudden after a few hours, the messages all of a sudden all got unblocked and started processing and finished in like minutes. When I looked at some sample messages from service log, I saw they are sent to SQS queue at like 8AM, and then it was never received until like 1PM....But this whole time SQS sent 0 messages back even though the SQS approximate visible message count was in the 15k range....Why would not a single message be returned for hours. Every message is using a unique messageGroupId btw.

Note that for the 700 inflight messages, I had manually set the visibility time out of those individual messages to some value like 17 minutes or so in the application (visibility timeout of each of those 700 messages individually, not the default visibility time out of the queue), but there shouldn't be any reason these 700 inflight messages would block the rest of the 15k messages because like I said they all have unique messageGroupId

I've looked online and have seen noone else with this issue. I am perplexed. Please help

  • Have you verified your SQS Delivery delay, Receive message wait time and visibility timeout configurations?

asked a year ago126 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions