How do I troubleshoot connectivity issues with my ElastiCache for Redis self-designed cluster?

7 minute read

I want to troubleshoot connectivity issues with my Amazon ElastiCache for Redis self-designed cluster.

Resolution

To troubleshoot connectivity issues with your ElastiCache for Redis self-designed cluster, complete the following:

Check that your cluster is in the Available state

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshoot AWS CLI errors. Also, make sure that you're using the most recent AWS CLI version.

If you recently created the cluster, then confirm that the cluster creation is complete and the cluster is ready to accept connections. To check the status of the cluster, use the ElastiCache console, the AWS CLI, or the ElastiCache API to view the cluster's details. Review the Status column and check for the following:

If the Status column shows Available, then the cluster is ready.
If the Status column shows Creating, then cluster creation is in progress. Wait a few minutes until the status updates to Available.
If the Status column shows Modifying, then the cluster's configuration is updating. Wait a few minutes until the modifications finish and the status updates to Available.

Check that your cluster is healthy

Nodes in a healthy cluster are in the Available state. To check the cluster's health, take the following actions:

Check the status of each node in your cluster.
Check event logs to see recent events in your cluster.
Check the Amazon CloudWatch metrics for an overview of the cluster's health. For more information, see Monitoring best practices with Amazon ElastiCache for Redis using Amazon CloudWatch.
Check the cluster and Redis client configurations to make sure that ElastiCache can perform recover and restore operations with minimal disruption. If you have hardware, instance, or software failures, then ElastiCache tries to recover and restore the service without user action. For more information, see Minimizing downtime in ElastiCache for Redis with Multi-AZ.

Check the network connectivity between your cluster and the client resource

To minimize latency, access ElastiCache from Amazon Elastic Compute Cloud (Amazon EC2) instances or from resources within the same Amazon Virtual Private Cloud (Amazon VPC). Also, you can connect from outside of the Amazon VPC or outside of AWS. For more information, see Access Patterns for accessing an ElastiCache cache in an Amazon VPC and Accessing your cluster or replication group. To analyze the connection and further troubleshoot connectivity issues between AWS resources, use the Amazon VPC Network Access Analyzer.

Check that the security groups and network access control lists (network ACLs) allow connections

Check the security groups and network ACLs on the cluster and resource that the connection is initiated from. Make sure that the security groups and network ACLs are configured correctly. The following are examples of client resources:

Amazon EC2 instances
AWS Lambda functions
Amazon Elastic Container Service (Amazon ECS) containers
Amazon Elastic Kubernetes Service (Amazon EKS) containers

Check the cluster security group

To check the cluster's security group, complete the following steps:

Open the ElastiCache console.
Select the cluster name from the Redis clusters menu, and then select the Network and security tab.
Note: By default, the cluster uses the Amazon VPC security group when no security group has been associated.
Check that at least one of the associated security groups allows inbound connections from the client resource to the cluster on the cluster's port.
To check the port number, check any of the cluster endpoints. The endpoints are in the domain_name:port format.
Note: By default, the cluster's port is 6379/TCP. You can override the port number when you create a cluster.
On the client resource, check that the security groups allow outbound connections to the cluster's port and to the CIDR blocks of the cluster's subnets.
Check that the network ACLs allow outgoing and incoming connections between the client and the cluster.
Note: By default, network ACLs allow all connections.
To further troubleshoot security group and network ACL configurations, use the Amazon VPC Network Access Analyzer.

Check that the correct endpoints are used for connections

Connection endpoints are recommended based on cluster configurations. For more information, see Finding connection endpoints.

Check the DNS resolution on the client side

If you have DNS issues, then you might receive a Name or service not known or NXDOMAIN error. To resolve these errors, make sure that you have the correct DNS attributes on the Amazon VPC client resource. It's a best practice to use the Amazon DNS server.

Check the TCP connection between Redis and the client

To establish and confirm connectivity between Redis and the client, use the curl command:
Note: Replace example-cluster-endpoint with your cluster's endpoint.

$ curl -v telnet://example-cluster-endpoint:6379

Example output:

*   Trying 172.31.1.242:6379...
* Connected to example-cluster-endpoint (172.31.1.242) port 6379

In the preceding example, the Connected keyword shows that the TCP connection is successful. If Connected doesn't appear in the command output, then check the following:

On the cluster, check that the security groups allow connections on the cluster's port. Confirm the port value on the cluster configuration page. (The default port is TCP/6379.) For more information, see Modifying an ElastiCache cluster.
On the cluster, check that all the shards and nodes are in the Available state. For more information, see Viewing a cluster's details.
On the client resource, check that the security groups allow outgoing connections to the cluster's IP address and port.
On the client resource, check that the routing tables have the appropriate routes and the cluster is reachable.
On the client resource, check that the correct connection is configured. The client resource can be in the same Amazon VPC, a different Amazon VPC, or outside of AWS. The connection can be configured to use a VPN, Amazon VPC peering, a transit Amazon VPC, AWS Transit Gateway, or AWS Direct Connect. For more information, see Access Patterns for accessing an ElastiCache cache in an Amazon VPC and Accessing your cluster or replication group.

Note: Amazon ElastiCache is designed to be accessed from the same Amazon VPC to make sure that there's low latency. Connections outside of Amazon VPC might increase latency. Increased latency occurs with connections that use the public internet directly or through tunneling. Also, increased latency might cause connectivity or timeout issues. If you blocked access, then use the Amazon VPC Reachability Analyzer to further troubleshoot.

Check the connections to clusters that have in-transit encryption

To send Redis traffic over TLS, use in-transit encryption. For a connection to be established, the client must have TLS support.

If you have redis-cli with TLS support, then add the --tls argument to the command:
Note: Replace example-encrypted-cluster-endpoint with your cluster's encrypted endpoint.

$ redis-cli -h example-encrypted-cluster-endpoint --tls

Example output:

example-encrypted-cluster-endpoint:6379>

If you have redis-cli without TLS support, then the following error appears:

"Unrecognized option or bad number of args for: '--tls'"

To troubleshoot the preceding error, install redis-cli with TLS support or use the openssl command for redis-cli without TLS support. For more information, see the Connect to a Redis cluster or replication group (Linux) section in Step 4: Connect to the cluster's node.

Check the connections to clusters that have authentication turned on

To check connections to clusters with authentication, use the redis-cli or openssl command. For more information, see the Connecting to an encryption/authentication enabled cluster section in Step 4: Connect to the cluster's node.

Note: The redis-cli command requires both the --tls and --askpass (or -a) arguments. If these arguments aren't provided or you use an incorrect password, then a NOAUTH or AUTH failed message appears. If you use redis-cli --tls, then use AUTH password after you connect to complete the authentication.