Why can't I reach my Amazon EMR cluster?

Resolution

Incorrect Amazon EMR security group rules

To resolve this issue, take the following actions:

• Verify that the Amazon EMR managed security group rules are correct for internal users and external users and applications.
• Allow TCP on port 8443 so that the cluster manager can communicate with the cluster's primary node.
• Allow SSH on port 22 so that you can use SSH to connect to the cluster.

Incorrect setup for Amazon EMR clusters in a private subnet

To resolve this issue, take the following actions:

• Add a managed security group for service access. The security group rules must allow the cluster manager to communicate with the cluster nodes.
• If you're using a bastion host, then configure the bastion host security group to allow inbound traffic from the client system. Also, configure the Amazon EMR cluster security groups to allow inbound traffic from the bastion host.

Note: Network configuration setups vary, so make sure that you correctly set the end-to-end connection without black holes.

Incorrect authentication setup

Check whether the authentication setup uses an Amazon Elastic Compute Cloud (Amazon EC2) keypair or Kerberos. Then, correctly configure your authentication setup.

Resource constraints in the cluster node

To resolve this issue, take the following actions:

• Verify that the underlying primary node exists and is in the running state.
• Check the instance-state log of the primary node to determine how you're using resources.
  Run the following command to check for the top CPU user:

  ps auxwww --sort -%cpu | head -10

  Run the following command to check the kernel's performance:

  dmesg | tail -n 25

  Run the following command to check memory usage:

  free -m

  Run the following command to check disk usage:

  df -h

Primary node's instance controller isn't running

Make sure that the primary node's instance controller is running. The instance controller is the daemon that runs on the cluster nodes and communicates with the Amazon EMR control plane and the rest of the cluster.

Run the following command to check the status of the instance controller on Amazon EMR versions 5.30.0 and later:

sudo systemctl status instance-controller

If the instance controller isn't active, then run the following command to start the instance controller:

sudo systemctl start instance-controller

To check the status of and start the instance controller on earlier Amazon EMR versions, see How do I restart a service in Amazon EMR?