How do I troubleshoot the READONLY error that I receive after a failover in my ElastiCache for Redis self-designed cluster?

Short description

If the primary node fails over to the replica nodes in your ElastiCache self-designed cluster, then the replica node becomes the primary node. This allows incoming requests to be served. The READONLY error occurs when you use a node endpoint instead of the primary endpoint of the cluster in your application. Also, this error occurs when DNS caching in the application routes traffic to the old primary node.

Resolution

To troubleshoot the READONLY error in your ElastiCache for Redis self-designed cluster that has cluster mode turned off, complete the following steps:

Check that cluster mode is turned off

Open the ElastiCache console, and then select Redis clusters. Check that the Cluster Mode for the cluster is turned off. For more information, see How do I resolve Redis client read requests that are redirected to the primary node of a shard instead of a replica node in ElastiCache for Redis?

Check the endpoint

Make sure that the primary endpoint is used and not the node endpoint. To check that the write commands go to the primary node, use one of the following methods:

Use redis-cli

1. Use redis-cli to the primary endpoint to connect to the cluster.

2. Set a key, and then read the key.
   Example:

   set key1 "hello"
   OK

3. Confirm that the key was set correctly.
   Example:

   get key1
   "hello"

Check the DNS setting

To prevent DNS caching issues, turn on retry logic in your application. Make sure that you follow the guidelines for the Redis client library that you use.

Check the incoming commands

To check incoming commands, use the MONITOR command. For more information, see MONITOR on the Redis website. To turn on MONITOR, connect to the cluster with redis-cli. The MONITOR command lists all the incoming commands to the cluster node.
Note: The MONITOR command is a resource expensive command and might severely decrease node performance. Use this command as a last option for troubleshooting.