EMR Studio creation error: The service role does not have permission to access the <CLUSTER>

0

Hi,

trying to attach a emr studio and workspace to a emr cluster via terraform. But get an error saying:

 Error: creating EMR Studio: InvalidRequestException: The service role does not have permission to access the dev-emr. Review the policies for the service role and the S3 bucket policy to ensure that the permission is granted. (Service: AWSEditors; Status Code: 400; Error Code: InvalidRequestException; Request ID: 9c904871-78e2-4893-88a4-f6410b3da4b0; Proxy: null)
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "9d6bd37a-c61f-4ad2-87e8-8eb9c4cb0f27"
│   },
│   Message_: "The service role does not have permission to access the dev-emr. Review the policies for the service role and the S3 bucket policy to ensure that the permission is granted. (Service: AWSEditors; Status Code: 400; Error Code: InvalidRequestException; Request ID: 9c904871-78e2-4893-88a4-f6410b3da4b0; Proxy: null)"
│ }
│ 
│   with aws_emr_studio.emr-studio,
│   on emr-studio.tf line 1, in resource "aws_emr_studio" "emr-studio":
│    1: resource "aws_emr_studio" "emr-studio" {

Studio code is:

resource "aws_emr_studio" "emr-studio" {
  name                        = "${local.full_name}-studio"
  auth_mode                   = "IAM"
  default_s3_location         = "s3://${var.s3_prevent_destroy == true ? aws_s3_bucket.emr[0].id : aws_s3_bucket.emr-destroy[0].id}/studio"
  engine_security_group_id    = aws_security_group.emr-studio.id
  service_role                = aws_iam_role.iam_emr_service_role.arn
  subnet_ids                  = [data.aws_subnets.intra.ids[0], data.aws_subnets.intra.ids[1], data.aws_subnets.intra.ids[2]]
  #user_role                   = aws_iam_role.iam_studio_user_role.arn
  vpc_id                      = data.aws_vpc.this.id
  workspace_security_group_id = aws_security_group.emr-studio.id
}

My iam role has

resource "aws_iam_role" "studio_service_role" {
  name = "${local.full_name}-studio-service-role"
  assume_role_policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
      {
        "Effect" : "Allow",
        "Principal" : {
          "Service" : "elasticmapreduce.amazonaws.com"
        },
        "Action" : "sts:AssumeRole",
    }]
  })
}

and the following policy attached:

resource "aws_iam_policy" "studio_service_role" {
  name        = "${local.full_name}-studio-service-policy"
  description = "copy of EMR_Notebooks_DefaultRole policy with extras"
  policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
       {
      "Sid": "AllowEMRReadOnlyActions",
      "Effect": "Allow",
      "Action": [
        "elasticmapreduce:ListInstances",
        "elasticmapreduce:DescribeCluster",
        "elasticmapreduce:ListSteps"
      ],
      "Resource": "*"
    },
    {
      "Sid": "AllowEC2ENIActionsWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateNetworkInterfacePermission",
        "ec2:DeleteNetworkInterface"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:network-interface/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
        }
      }
    },
    {
      "Sid": "AllowEC2ENIAttributeAction",
      "Effect": "Allow",
      "Action": [
        "ec2:ModifyNetworkInterfaceAttribute"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:instance/*",
        "arn:aws:ec2:*:*:network-interface/*",
        "arn:aws:ec2:*:*:security-group/*"
      ]
    },
    {
      "Sid": "AllowEC2SecurityGroupActionsWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:AuthorizeSecurityGroupEgress",
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:RevokeSecurityGroupEgress",
        "ec2:RevokeSecurityGroupIngress",
        "ec2:DeleteNetworkInterfacePermission"
      ],
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
        }
      }
    },
    {
      "Sid": "AllowDefaultEC2SecurityGroupsCreationWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateSecurityGroup"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:security-group/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
        }
      }
    },
    {
      "Sid": "AllowDefaultEC2SecurityGroupsCreationInVPCWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateSecurityGroup"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:vpc/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
        }
      }
    },
    {
      "Sid": "AllowAddingEMRTagsDuringDefaultSecurityGroupCreation",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateTags"
      ],
      "Resource": "arn:aws:ec2:*:*:security-group/*",
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true",
          "ec2:CreateAction": "CreateSecurityGroup"
        }
      }
    },
    {
      "Sid": "AllowEC2ENICreationWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateNetworkInterface"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:network-interface/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
        }
      }
    },
    {
      "Sid": "AllowEC2ENICreationInSubnetAndSecurityGroupWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateNetworkInterface"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:subnet/*",
        "arn:aws:ec2:*:*:security-group/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
        }
      }
    },
    {
      "Sid": "AllowAddingTagsDuringEC2ENICreation",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateTags"
      ],
      "Resource": "arn:aws:ec2:*:*:network-interface/*",
      "Condition": {
        "StringEquals": {
          "ec2:CreateAction": "CreateNetworkInterface"
        }
      }
    },
    {
      "Sid": "AllowEC2ReadOnlyActions",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeNetworkInterfaces",
        "ec2:DescribeTags",
        "ec2:DescribeInstances",
        "ec2:DescribeSubnets",
        "ec2:DescribeVpcs"
      ],
      "Resource": "*"
    },
    {
      "Sid": "AllowSecretsManagerReadOnlyActionsWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue"
      ],
      "Resource": "arn:aws:secretsmanager:*:*:secret:*",
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
        }
      }
    },
    {
      "Sid": "AllowWorkspaceCollaboration",
      "Effect": "Allow",
      "Action": [
        "iam:GetUser",
        "iam:GetRole",
        "iam:ListUsers",
        "iam:ListRoles",
        "sso:GetManagedApplicationInstance",
        "sso-directory:SearchUsers"
      ],
      "Resource": "*"
      },
      {
        "Effect" : "Allow",
        "Action" : "secretsmanager:GetSecretValue",
        "Resource" : "*"
      },
      {
        "Effect" : "Allow",
        "Action" : "s3:*",
        "Resource" : "*"
      },
      {
        "Effect" : "Allow",
        "Action" : [
          "ec2:AuthorizeSecurityGroupEgress",
          "ec2:AuthorizeSecurityGroupIngress",
          "ec2:CreateSecurityGroup",
          "ec2:DescribeSecurityGroups",
          "ec2:RevokeSecurityGroupEgress",
          "ec2:CreateNetworkInterface",
          "ec2:CreateNetworkInterfacePermission",
          "ec2:DeleteNetworkInterface",
          "ec2:DeleteNetworkInterfacePermission",
          "ec2:DescribeNetworkInterfaces",
          "ec2:ModifyNetworkInterfaceAttribute",
          "ec2:DescribeTags",
          "ec2:DescribeInstances",
          "ec2:DescribeSubnets",
          "ec2:DescribeVpcs",
          "elasticmapreduce:ListInstances",
          "elasticmapreduce:DescribeCluster",
          "elasticmapreduce:ListSteps"
        ],
        "Resource" : "*"
      },
      {
        "Effect" : "Allow",
        "Action" : "ec2:CreateTags",
        "Resource" : "arn:aws:ec2:*:*:network-interface/*",
        "Condition" : {
          "ForAllValues:StringEquals" : {
            "aws:TagKeys" : [
              "aws:elasticmapreduce:editor-id",
              "aws:elasticmapreduce:job-flow-id"
            ]
          }
        }
      }
    ]
  })
}

this is basically a copy and paste of all resource options from the following docs:

https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio-service-role.html

https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-notebooks-service-role.html

Any idea what im doing wrong?

demandé il y a un an1859 vues
5 réponses
2

In your Terraform configuration, after creating the IAM role and policy, you also need to create an IAM role policy attachment. This resource links the policy to the role. Here's how you might do it:

resource "aws_iam_role_policy_attachment" "studio_service_role_attachment" {
  role       = aws_iam_role.studio_service_role.name
  policy_arn = aws_iam_policy.studio_service_role.arn
}

In your Terraform configuration, after creating the IAM role and policy, you also need to create an IAM role policy attachment. This resource links the policy to the role. Here's how you might do it:

resource "aws_iam_role_policy_attachment" "studio_service_role_attachment" {
  role       = aws_iam_role.studio_service_role.name
  policy_arn = aws_iam_policy.studio_service_role.arn
}
profile picture
EXPERT
répondu il y a un an
0

Hi, @sdtslmn

Sorry yes i have attached the policy to the role, I also found that i have to remove all the tagging logic as I wont be adding tags to all the resources needed to match that pattern.

My iam-studio-service-role.tf now looks like below:

# More info for IAM can be found:
# https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio-service-role.html
# https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-notebooks-service-role.html

resource "aws_iam_role" "studio_service_role" {
  name = "${local.full_name}-studio-service-role"
  assume_role_policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
      {
        "Effect" : "Allow",
        "Principal" : {
          "Service" : "elasticmapreduce.amazonaws.com"
        },
        "Action" : "sts:AssumeRole",
    }]
  })
}

resource "aws_iam_role_policy_attachment" "studio_service_role" {
  role       = aws_iam_role.studio_service_role.name
  policy_arn = aws_iam_policy.studio_service_role.arn
}

resource "aws_iam_policy" "studio_service_role" {
  name        = "${local.full_name}-studio-service-policy"
  description = "copy of EMR_Notebooks_DefaultRole policy with extras"
  policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
       {
      "Sid": "AllowEMRReadOnlyActions",
      "Effect": "Allow",
      "Action": [
        "elasticmapreduce:ListInstances",
        "elasticmapreduce:DescribeCluster",
        "elasticmapreduce:ListSteps"
      ],
      "Resource": "*"
    },
    {
      "Sid": "AllowEC2ENIActionsWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateNetworkInterfacePermission",
        "ec2:DeleteNetworkInterface"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:network-interface/*"
      ],
    },
    {
      "Sid": "AllowEC2ENIAttributeAction",
      "Effect": "Allow",
      "Action": [
        "ec2:ModifyNetworkInterfaceAttribute"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:instance/*",
        "arn:aws:ec2:*:*:network-interface/*",
        "arn:aws:ec2:*:*:security-group/*"
      ]
    },
    {
      "Sid": "AllowEC2SecurityGroupActionsWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:AuthorizeSecurityGroupEgress",
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:RevokeSecurityGroupEgress",
        "ec2:RevokeSecurityGroupIngress",
        "ec2:DeleteNetworkInterfacePermission"
      ],
      "Resource": "*",
    },
    {
      "Sid": "AllowDefaultEC2SecurityGroupsCreationWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateSecurityGroup"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:security-group/*"
      ],
    },
    {
      "Sid": "AllowDefaultEC2SecurityGroupsCreationInVPCWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateSecurityGroup"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:vpc/*"
      ],
    },
    {
      "Sid": "AllowAddingEMRTagsDuringDefaultSecurityGroupCreation",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateTags"
      ],
      "Resource": "arn:aws:ec2:*:*:security-group/*",
    },
    {
      "Sid": "AllowEC2ENICreationWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateNetworkInterface"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:network-interface/*"
      ],
    },
    {
      "Sid": "AllowEC2ENICreationInSubnetAndSecurityGroupWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateNetworkInterface"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:subnet/*",
        "arn:aws:ec2:*:*:security-group/*"
      ],
    },
    {
      "Sid": "AllowAddingTagsDuringEC2ENICreation",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateTags"
      ],
      "Resource": "arn:aws:ec2:*:*:network-interface/*",
    },
    {
      "Sid": "AllowEC2ReadOnlyActions",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeNetworkInterfaces",
        "ec2:DescribeTags",
        "ec2:DescribeInstances",
        "ec2:DescribeSubnets",
        "ec2:DescribeVpcs"
      ],
      "Resource": "*"
    },
    {
      "Sid": "AllowSecretsManagerReadOnlyActionsWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue"
      ],
      "Resource": "arn:aws:secretsmanager:*:*:secret:*",
    },
    {
      "Sid": "AllowWorkspaceCollaboration",
      "Effect": "Allow",
      "Action": [
        "iam:GetUser",
        "iam:GetRole",
        "iam:ListUsers",
        "iam:ListRoles",
        "sso:GetManagedApplicationInstance",
        "sso-directory:SearchUsers"
      ],
      "Resource": "*"
      },
      {
        "Effect" : "Allow",
        "Action" : "secretsmanager:GetSecretValue",
        "Resource" : "*"
      },
      {
        "Effect" : "Allow",
        "Action" : "s3:*",
        "Resource" : "*"
      },
      {
        "Effect" : "Allow",
        "Action" : [
          "ec2:AuthorizeSecurityGroupEgress",
          "ec2:AuthorizeSecurityGroupIngress",
          "ec2:CreateSecurityGroup",
          "ec2:DescribeSecurityGroups",
          "ec2:RevokeSecurityGroupEgress",
          "ec2:CreateNetworkInterface",
          "ec2:CreateNetworkInterfacePermission",
          "ec2:DeleteNetworkInterface",
          "ec2:DeleteNetworkInterfacePermission",
          "ec2:DescribeNetworkInterfaces",
          "ec2:ModifyNetworkInterfaceAttribute",
          "ec2:DescribeTags",
          "ec2:DescribeInstances",
          "ec2:DescribeSubnets",
          "ec2:DescribeVpcs",
          "elasticmapreduce:ListInstances",
          "elasticmapreduce:DescribeCluster",
          "elasticmapreduce:ListSteps"
        ],
        "Resource" : "*"
      },
      {
        "Effect" : "Allow",
        "Action" : "ec2:CreateTags",
        "Resource" : "arn:aws:ec2:*:*:network-interface/*",
      }
    ]
  })
}

And i still get the error:

Error: creating EMR Studio: InvalidRequestException: The service role does not have permission to access the oc-dev-emr. Review the policies for the service role and the S3 bucket policy to ensure that the permission is granted. (Service: AWSEditors; Status Code: 400; Error Code: InvalidRequestException; Request ID: 38f2aa57-7ba0-48e0-899d-15c13db8c4f1; Proxy: null)
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "c8a17a22-609a-41b8-ac86-59950e786dbe"
│   },
│   Message_: "The service role does not have permission to access the oc-dev-emr. Review the policies for the service role and the S3 bucket policy to ensure that the permission is granted. (Service: AWSEditors; Status Code: 400; Error Code: InvalidRequestException; Request ID: 38f2aa57-7ba0-48e0-899d-15c13db8c4f1; Proxy: null)"
│ }
│ 
│   with aws_emr_studio.emr-studio,
│   on emr-studio.tf line 1, in resource "aws_emr_studio" "emr-studio":
│    1: resource "aws_emr_studio" "emr-studio" {
│ 

To add to my confusion! if i create a studio via the console using the same IAM role, it works successufully and lets bring up a workspace

répondu il y a un an
0

I am also having the same issues, any further update/resolution on this please? Thanks Suddhasil

répondu il y a 9 mois
0

for me below policy worked :

resource "aws_iam_policy" "emr_studio_policy" {
  name        = "emr_studio_policy"
  description = "Allow all accesses needed for EMR"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
          {
          "Effect": "Allow",
          "Action": [
            "elasticmapreduce:*",
            "ec2:CreateNetworkInterfacePermission",
            "ec2:DeleteNetworkInterface"
          ],
          "Resource": "*"
        },
        {
          "Effect": "Allow",
          "Action": "elasticmapreduce:StartNotebookExecution",
          "Resource": "arn:aws:elasticmapreduce:us-east-1:account_no:notebook-instance/*"
        },
        {
          "Effect": "Allow",
          "Action": [
            "s3:GetObject",
            "s3:PutObject"
          ],
          "Resource": [
            local.bucket_arn_with_wildcard
          ]
        },
        {
          "Effect": "Allow",
          "Action": "s3:ListBucket",
          "Resource": var.s3_bucket_arn
        }
    ]
  })
}

this is the policy code, and then had to attach this policy to the role created.

Chetan
répondu il y a 8 mois
0

This is my IAM role I succeed for EMR Studio. Hope it helps!

resource "aws_iam_role" "hm_amazon_emr_studio_iam_role" {
  name = "AmazonEMRStudioServiceRole-${var.amazon_emr_studio_name}"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = "sts:AssumeRole"
        Principal = {
          Service = "elasticmapreduce.amazonaws.com"
        }
      }
    ]
  })
  tags = {
    Environment = var.environment
    Team        = var.team
    Name        = "AmazonEMRStudioServiceRole-${var.amazon_emr_studio_name}"
  }
}
resource "aws_iam_role_policy" "hm_amazon_emr_studio_iam_role_input_policy" {
  name = "AmazonEMRStudioServicePolicyForS3-${var.amazon_emr_studio_name}"
  role = aws_iam_role.hm_amazon_emr_studio_iam_role.name
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "s3:DeleteObject",
          "s3:GetEncryptionConfiguration",
          "s3:GetObject",
          "s3:ListBucket",
          "s3:PutObject"
        ]
        Resource = [
          "arn:aws:s3:::${var.s3_bucket}",
          "arn:aws:s3:::${var.s3_bucket}/*"
        ]
      },
      {
        Effect = "Allow"
        Action = [
          "s3:ListAllMyBuckets"
        ]
        Resource = [
          "*"
        ]
      }
    ]
  })
}
resource "aws_iam_role_policy_attachment" "hm_amazon_emr_studio_iam_role_policy_attachment" {
  role       = aws_iam_role.hm_amazon_emr_studio_iam_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceEditorsRole"
}
profile picture
répondu il y a 7 mois

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions