An SNS message with a double quote in ANY attribute is filtered out. Is this a bug?

Question

The documentation https://docs.aws.amazon.com/sns/latest/dg/sns-subscription-filter-policies.html says
> When Amazon SNS evaluates message attributes against the subscription filter policy, it ignores message attributes that aren't specified in the policy.

Through some experimentation I have determined that SNS messages with a double quote in one of their attribute values will always be filtered out for a subscription with a filter policy, even if the filter policy doesn't mention the attribute. It does NOT get filtered out if you put a backslash character before the double quote. I can't find any reference to this in the documentation so I don't know if it's expected behavior. It seems like a bug.

Example:

SNS subscription filter policy:
```
{
    "action": ["purchase"]
}
```
Message Attributes (PASSES filter):
* action - DataType: String, StringValue: purchase
* metadata - DataType: String, StringValue: abc

Message Attributes (DOES NOT PASS filter):
* action - DataType: String, StringValue: purchase
* metadata - DataType: String, StringValue: "abc

Message Attributes (PASSES filter):
* action - DataType: String, StringValue: purchase
* metadata - DataType: String, StringValue: \"abc

This problem occurred because we have a String type attribute with a JSON blob in the value. I eventually narrowed the problem down to just the presence of the double quote character.

I have found a few examples of other people encountering this issue:
* https://forums.aws.amazon.com/thread.jspa?threadID=300797
* https://forums.aws.amazon.com/thread.jspa?threadID=318176
* https://forums.aws.amazon.com/message.jspa?messageID=841824

Is this a bug?

Answer

Hello,

Thank you for reaching out to us. SNS service team is aware of the issue with few characters that might cause an issue with filter policy. When there is filter policy associated with a subscription and the published messages contains attributes with escapable characters such as " and \, SNS expects the characters to be double-escaped otherwise the message filtering is affected thereby causing the message delivery failure to the endpoint.

Please find the list of such special characters:

======================
      Escape Characters List
======================

1. "
2. \
3. /
4. b
5. f
6. n
7. r
8. t
9. uXXXX (where X is hex eg. u123a)

For example:

Character in escape list:

\” bad

\” ok

\\” bad

Character not in escape list:

\p ok

\p bad

\\p ok

Internal service team is working towards a fix and I am unable to provide an ETA for this , updates to the services are announced at AWS What's New [1] and RSS feed [2].

References:

1.  https://aws.amazon.com/new/
2.  https://aws.amazon.com/new/feed/

An SNS message with a double quote in ANY attribute is filtered out. Is this a bug?

====================== Escape Characters List

Relevant content