EMR: HBase cluster WAL writer failure

Question

I'm trying to create a EMR 7.1.0 cluster with HBase enabled for full S3 backup (including WAL) via the web console. However, no AWSServiceRoleForEMRWAL role is automatically being created and thus my hbase cannot obtain the WAL log writer because the instance doesn't have permissions to S3.  Both my service role and my instance role used by the cluster contain the iam:CreateServiceLinkedRole, iam:PutRolePolicy, and iam:UpdateRoleDescription permissions.  Yet still no love.  I've reviewed all the cluster logs and cannot find any error pertaining to AWSServiceRoleForEMRWAL creation failure. Any advice on how to troubleshoot or otherwise encourage EMR to auto create the AWSServiceRoleForEMRWAL role as outlined here https://docs.aws.amazon.com/emr/latest/ManagementGuide/using-service-linked-roles-wal.html.

Yokesh NK · Answer

Hello,

For enabling EMR WAL below are the pre-requsites which are required as mentioned **[here](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hbase-wal-permissions.html)**

Please add the below policies to EMR-EC2 instance profile role if you have not added yet. Also, just in case missed, please ensure that your region is part of [supported list. ](https://docs.aws.amazon.com/emr/latest/ManagementGuide/using-service-linked-roles-wal.html#emr-slr-regions-wal)

```
Policy 1 - 
==================== 
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Action": [
                "emrwal:DeleteWal",
                "emrwal:CreateWal",
                "emrwal:CreateWorkspace",
                "emrwal:AppendEdit",
                "emrwal:ReplayEdits",
                "emrwal:GetCurrentWalTime",
                "emrwal:CompleteWalFlush"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}
==================== 
```
```
Policy 2 - 
==================== 
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:CreateServiceLinkedRole",
                "iam:PutRolePolicy"
            ],
            "Resource": "arn:aws:iam::*:role/aws-service-role/emrwal.amazonaws.com*/AWSServiceRoleForEMRWAL*",
            "Condition": {
                "StringLike": {
                    "iam:AWSServiceName": [
                        "emrwal.amazonaws.com",
                        "elasticmapreduce.amazonaws.com.cn"
                    ]
                }
            }
        }
    ]
}
====================
```

donaldthecat · Answer

I can't create that first policy.  I get an error for all the Actions, saying:

Invalid Service In Action: The service emrwal:DeleteWal specified in the action does not exist.

Those don't appear to be valid services that I'm allowed to put in the policy.  My understanding was that the AWSServiceRoleForEMRWAL service role would automatically have those emrwal:* service permissions and that the AWSServiceRoleForEMRWAL would be automatically created for me because I have Policy #2 already attached to my ec2 instances.  However, that role isn't being created.

EMR: HBase cluster WAL writer failure

Relevant content