How do I implement Redis keyspace notifications in ElastiCache?

Short description

Redis keyspace notifications allow you to subscribe to PubSub channels. Through these channels, published events are received when a Redis command or data alteration occurs. These notifications are useful when an application must respond to changes that occur to the value stored in specific keys. Also, you can use Redis keyspace notifications to follow dataset changes. The keyspace notifications feature is available in versions 2.8.6 and later. When a Redis keyspace notification is invoked, a keyspace notification and key-event notification occur. The keyspace channel receives the name of the event, and the key-event channel receives the name of the key as a message.

Resolution

Note: The following resolution applies to ElastiCache clusters with cluster mode turned off. For more information, see Redis keyspace notifications on the Redis website.

To implement Redis keyspace notifications in ElastiCache, complete the following steps:

Activate Redis keyspace notifications

Note: By default, ElastiCache turns off Redis keyspace notifications. To activate keyspace notifications in a custom cache parameter group, use the notify-keyspace-events parameter. This parameter value uses multiple parameters to determine which channel (keyspace or key-event) is used and the information to post to the channel. For more information, see Redis 2.8.6 added parameters.

1. Open the ElastiCache console.
2. To see a list of the available parameter groups, choose Parameter Groups in the navigation pane.
3. Select the parameter group that you want to modify.
   Note: You can't modify a default parameter group. If all the parameter groups listed are default, then create a new parameter group.
4. (Optional) To create a new parameter group, choose Create Parameter Group:
   For the Name field, enter a unique name.
   For the Description field, enter a description.
   Select a Family.
   Choose Create.
5. Choose Edit parameters values.
6. Navigate to notify-keyspace-events, and then enter AKE in the Value field. For more information about allowed values, see Redis 2.8.6 added parameters.
7. Choose Save changes. The changes take effect immediately and the cluster doesn't need to be restarted.
   Note: Be sure to assign the new modified parameter group to your Redis cluster.

Generate events

To generate events, use either the Redis Command Line Interface (CLI) or the Redis Python client:

Use the Redis CLI (redis-cli)

1. To subscribe to notifications, use redis-cli to run the following command:

   #./redis-cli -h testdisabled.xxxxx.ng.0001.use1.cache.amazonaws.com -p 6379 --csv psubscribe '__key*__:*'

2. To create keyspace events, run any set of commands. The following example commands set the value and expire time to 3 seconds for key Redis and subscriber for expired events.

   #./redis-cli -h testdisabled.xxxxx.ng.0001.use1.cache.amazonaws.com -p 6379
   testdisabled.xm4oz6.ng.0001.use1.cache.amazonaws.com:6379>
   testdisabled.xm4oz6.ng.0001.use1.cache.amazonaws.com:6379> SET key1 value1 ex 5
   testdisabled.xm4oz6.ng.0001.use1.cache.amazonaws.com:6379>

3. Confirm that the events generate. The following is an example output from the generated events:

   #./redis-cli -h testdisabled.xxxxx.ng.0001.use1.cache.amazonaws.com -p 6379 --csv psubscribe '__key*__:*' 
   Reading messages... (press Ctrl-C to quit)
   "psubscribe","__key*__:*",1
   "pmessage","__key*__:*","__keyspace@0__:key1","set"
   "pmessage","__key*__:*","__keyevent@0__:set","key1"
   "pmessage","__key*__:*","__keyspace@0__:key1","expire"
   "pmessage","__key*__:*","__keyevent@0__:expire","key1"
   "pmessage","__key*__:*","__keyspace@0__:key1","expired"
   "pmessage","__key*__:*","__keyevent@0__:expired","key1"

   For the preceding example output, the first event indicates that you successfully subscribed to 1 channel. The second event is a keyspace notification that indicates the event name set as the message. The third event is a key-event notification that indicates the key name key1 as the message. After the third event, you can see expire messages for the keyspace and key-event notifications.

Use the Redis Python client (redis-py)

1. To subscribe to notifications, use a Python 2 or Python 3 script.
   Note: The following scripts subscribe to all keyspace notifications and print received events.
   Python 2:

   # cat subscribe.py 
   import redis
   import datetime
   import time
   r = redis.Redis(host='testdisabled.xxxxx.ng.0001.use1.cache.amazonaws.com', port=6379, db=0)
   #Subscribing for events matching pattern "__key*__:*"
   p = r.pubsub()
   p.psubscribe('__key*__:*')
   print('Starting message loop')
   while True:
       message = p.get_message()
       if message:
           print datetime.datetime.now(), message
       else:
           time.sleep(0.01)

   Python 3:

   import redis
   from datetime import datetime
   import time
   r = redis.Redis(host = 'testdisabled.xxxxx.ng.0001.use1.cache.amazonaws.com', port = 6379, db = 0)
   #Subscribing for events matching pattern "__key*__:*
   p = r.pubsub()
   p.psubscribe('__key*__:*')
   print('Starting message loop')
   while True:
       message = p.get_message()
       if message:
           print (datetime.now(), message)
       else:
           time.sleep(0.01)

   Note: Server overload might occur when lower values or the absence of sleep causes the client to constantly poll the channel. Higher values cause increased memory usage on the server. The preceding scripts set the time.sleep to 0.01.

2. In a separate session, connect to the cluster, and then run the createkey.py script to set or expire keys:
   Note: Make sure that you set the example-key as key and example-value as value. Also, use a value of 3 seconds for the expire time.

   # cat createkey.py
   import redis
   r = redis.Redis(host='testdisabled.xxxxx.ng.0001.use1.cache.amazonaws.com', port=6379, db=0)
   r.set('example-key','example-value')
   r.expire('example-key',3)

3. In a separate session, run the subscribe.py script.
   Example output:

   # python2 subscribe.py
   Starting message loop
   2024-04-29 23:40:56.356659 {u'pattern': None, u'type': 'psubscribe', u'channel': '__key*__:*', u'data': 1L}
   2024-04-29 23:40:56.356659 {u'pattern': None, u'type': 'psubscribe', u'channel': '__key*__:*', u'data': 1L}
   2024-04-29 23:41:00.016410 {u'pattern': '__key*__:*', u'type': 'pmessage', u'channel': '__keyspace@0__:example-key', u'data': 'set'}
   2024-04-29 23:41:00.016512 {u'pattern': '__key*__:*', u'type': 'pmessage', u'channel': '__keyevent@0__:set', u'data': 'example-key'}
   2024-04-29 23:41:00.016582 {u'pattern': '__key*__:*', u'type': 'pmessage', u'channel': '__keyspace@0__:example-key', u'data': 'expire'}
   2024-04-29 23:41:00.016650 {u'pattern': '__key*__:*', u'type': 'pmessage', u'channel': '__keyevent@0__:expire', u'data': 'example-key'}
   2024-04-29 23:41:03.111509 {u'pattern': '__key*__:*', u'type': 'pmessage', u'channel': '__keyspace@0__:example-key', u'data': 'expired'}
   2024-04-29 23:41:03.111601 {u'pattern': '__key*__:*', u'type': 'pmessage', u'channel': '__keyevent@0__:expired', u'data': 'example-key'}

   Note: After you complete the preceding command, your Redis key listener is ready.