How do I troubleshoot Application Load Balancer session stickiness issues?

Short description

Sticky sessions use cookies to help the client maintain a connection to the same target over a cookie's lifetime. Sticky sessions configure a load balancer to bind user sessions to a specific target. This means that all requests from a user during a session are sent to the same target. However, this assumption about the connection can cause imbalances over time.

A sticky session can fail for the following reasons:

• The registered target didn't generate a cookie.
• The client didn't return the cookie in the request header.
• The cookies aren't formatted correctly.
• The duration-based session is over.
• The session request passes through multiple load balancers.
• The target health status changed to unhealthy.
• An AWS service turned off stickiness.

Note: Before you begin, make sure that you review the requirements and considerations sections in Sticky sessions for your Application Load Balancer.

Resolution

Application-based session stickiness

1. Check for HTTP errors with your load balancer. For instructions, see Troubleshoot your Application Load Balancers.

2. To check for cookies placed on the backend instance or load balancer, run a curl command similar to the following. Replace the DNS name with your load balancer name:

   [ec2-user@ip-172-31-22-85 ~]$ curl -vko /dev/null internal-EXAMPLE-ELB-1430759361.eu-central-1.elb.amazonaws.com

   Note: You can also install the Linux curl utility on the Windows operating system (OS). For more information, see curl 8.10.1 for Windows on the curl website.

   The output from the curl command looks similar to the following:

   ...
   < Set-Cookie: PHPSESSID=k0qu6t4e35i4lgmsk78mj9k4a4; path=/
   
   < Set-Cookie:
   
   AWSALBAPP-0=438DC7A50C516D797550CF7DE2A7DBA19D6816D5E6FB20329CD6AEF2B40030B12FF2839757A60E2330136A2182D27D049FB9D887FBFE9E80FB0724130FB3A86A4B0BAC296FDEB9E943EC9272FF52F5A8AEF373DF33;PATH=/
   
   ...

3. To verify that the registered target generated an application cookie, send an HTTP request to the instance IP address similar to the following:

   [ec2-user@ip-172-31-22-85 ~]$ curl -vko /dev/null 172.31.30.168

   The output from the command looks similar to the following:

   ...
   < Set-Cookie: PHPSESSID=5pq74110nuir60kpapj04mglg4; path=/
   
   ...

4. Verify that the cookie name generated by the registered target is the name of the cookie on the load balancer from steps 2 and 3.

5. When the target generates an application cookie and the load balancer generates an AWSALBAPP cookie, check that the client sends both cookies in subsequent requests. If the client doesn't include either the application or the AWSELB cookie, then stickiness fails. To verify that the client sends both the application and AWSALBAPP cookie, take a packet capture on the client. To retrieve the cookie information in the request header, use one of the following options:

   tcpdump from the tcpdump website
   Wireshark utility from the Wireshark website
   Browser web-debugging tool

   Note: If you are working with AWS Support, then create a HAR file to gather this information. Because HAR files can capture sensitive information, such as usernames, passwords, and keys, make sure that you remove sensitive information from a HAR file.

6. Check the backend instance that the load balancer routed the request to. To use instance metadata to display the instance ID, run a script similar to the following:

   <?php $instance_id =file_get_contents('http://169.254.169.254/latest/meta-data/instance-id');echo "instance id = " . $instance_id . "\\xA";?>

7. To check whether requests from the same user are routed to different registered targets, review your Elastic Load Balancing (ELB) access logs. For instructions, see Access logs for your Application Load Balancer.

8. Verify that the health status of all targets under the target group where stickiness is turned on are healthy. If the target health status changes to unhealthy, then stickiness breaks and the load balancer doesn't route requests to that target. A new healthy target is then automatically selected by the load balancer and establishes a sticky session. The load balancer continues to route requests to the new target even if the health status is unhealthy. For more information on health checks, see Health checks for Application Load Balancer target groups.

9. Check for AWS services, such as Amazon Elastic Kubernetes Service (Amazon EKS), that might have turned off the stickiness in your load balancer. Check the CloudTrail Event history. Use the API name ModifyTargetGroupAttributes and the attribute stickiness.enabled.

Duration-based session stickiness

1. To check for an AWSALB cookie, run a curl command similar to the following. Make sure that you use the load balancer DNS name:

   [ec2-user@ip-172-31-22-85 ~]$ curl -vko /dev/null internal-TESTELB-1430759361.eu-central-1.elb.amazonaws.com

   The output from the curl command looks similar to the following:

   ...
   < Set-Cookie: AWSALB=438DC7A50C516D797550CF7DE2A7DBA19D6816D5E6FB20329CD6AEF2B40030B12FF2839757A60E2330136A2182D27D049FB9D887FBFE9E80FB0724130FB3A86A4B0BAC296FDEB9E943EC9272FF52F5A8AEF373DF33;PATH=/
   ...

   Note: If there isn't an AWSALB cookie in the response, then there is no stickiness between the client and the backend instance.

2. If the load balancer generated an AWSALB cookie, then check that the client sends this cookie in subsequent requests. If the client fails to include the AWSALB cookie, then stickiness doesn't work. To verify that the client sends the AWSALB cookie, take a packet capture on the client. To retrieve the cookie information in the request header, use one of the following options:

   tcpdump from the tcpdump website
   Wireshark utility from the Wireshark website
   Browser web-debugging tool

   Note: If you are working with AWS Support, then create a HAR file to gather this information. Because HAR files can capture sensitive information, such as usernames, passwords, and keys, make sure that you remove sensitive information from a HAR file.

3. Check the duration configured on the load balancer. If the cookie expiration passed, then client sessions no longer stick to the registered target until a new cookie is issued by the load balancer.

4. If your request passes through multiple load balancers, then verify that stickiness is activated on only one load balancer. If more than one load balancer generates a cookie, then the load balancer replaces the original cookie and stickiness fails.

For Classic Load Balancers, see How can I troubleshoot the Classic Load Balancer session stickiness feature?

Related information

Why is Elastic Load Balancing unequally routing my load balancer traffic?

Configure sticky sessions for your Classic Load Balancer

How do I set up weighted target groups for my Application Load Balancer?