SQS FIFO Queue Deduplication

Question

I am a bit confused by the meaning of "any messages sent with the same message deduplication ID are accepted successfully but aren't delivered during the 5-minute deduplication interval."

I send message A with `deduplication id` of 1234.  I send another message B with the same `deduplication id` of 1234.

Does SQS ignore the second message B. Or save it and makes it visible after the 5-minute deduplication interval?

Answer

Hello.

I tried it in my environment.  
``` bash
for i in {0..2}; do
    aws sqs send-message \
        --queue-url ${QUEUE_URL} \
        --message-body '{"message": "fifo-queue-duplicate-test"}' \
        --message-group-id Group1 \
        --message-deduplication-id "1"
done
```
When you run the above command, three messages will be sent to the SQS queue.  
As a result, I was able to send three messages, but only one could be displayed as a message.  
If you send a message with the same ID(above command) after 5 minutes, you can now see two messages.  
In other words, I believe that the same ID will not be deduplicated if 5 minutes have elapsed.

Answer

Thanks - that's what I'm seeing. So deduplication is only with 5 minutes, the  messages will still be deduplicated after the 5 minutes is up.

I'm guessing there isn't any configuration that can be changed that if the  same deduplication id is used, to discard the duplicate messages?

Answer

If you send a message with the same dedup ID, within 5 minutes of the previous message, the new message is accepted by the queue, but discarded. You will only be able to consume it once.

Answer

okay - that was my expected behaviour but not what I've noticed. I will investigate further and see whats happening.

SQS FIFO Queue Deduplication

Relevant content