SQS FIFO Queue w/ every message having its own unique message group ID is long polling for 0 when queue has more than 10k items

0

I have a FIFO queue setup where each single message sent is sent with a guaranteed unique message group ID.

DeduplicationScope = perMessageGroup FifoThroughputLimit=perMessageGroupId

I've been using this queue fine for months until today when something very strange happened.

At one point I had about 700 messages in flight and around 15k messages visible in queue. At this point when I made a recieveMessage request with with long polling with a message return count of 10, i was getting zero messages returned consistently for like hours. Then randomly all of a sudden after a few hours, the messages all of a sudden all got unblocked and started processing and finished in like minutes. When I looked at some sample messages from service log, I saw they are sent to SQS queue at like 8AM, and then it was never received until like 1PM....But this whole time SQS sent 0 messages back even though the SQS approximate visible message count was in the 15k range....Why would not a single message be returned for hours. Every message is using a unique messageGroupId btw.

Note that for the 700 inflight messages, I had manually set the visibility time out of those individual messages to some value like 17 minutes or so in the application (visibility timeout of each of those 700 messages individually, not the default visibility time out of the queue), but there shouldn't be any reason these 700 inflight messages would block the rest of the 15k messages because like I said they all have unique messageGroupId

I've looked online and have seen noone else with this issue. I am perplexed. Please help

  • Have you verified your SQS Delivery delay, Receive message wait time and visibility timeout configurations?

질문됨 일 년 전131회 조회
답변 없음

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠