EMR Studio creation error: The service role does not have permission to access the <CLUSTER>

0

Hi,

trying to attach a emr studio and workspace to a emr cluster via terraform. But get an error saying:

 Error: creating EMR Studio: InvalidRequestException: The service role does not have permission to access the dev-emr. Review the policies for the service role and the S3 bucket policy to ensure that the permission is granted. (Service: AWSEditors; Status Code: 400; Error Code: InvalidRequestException; Request ID: 9c904871-78e2-4893-88a4-f6410b3da4b0; Proxy: null)
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "9d6bd37a-c61f-4ad2-87e8-8eb9c4cb0f27"
│   },
│   Message_: "The service role does not have permission to access the dev-emr. Review the policies for the service role and the S3 bucket policy to ensure that the permission is granted. (Service: AWSEditors; Status Code: 400; Error Code: InvalidRequestException; Request ID: 9c904871-78e2-4893-88a4-f6410b3da4b0; Proxy: null)"
│ }
│ 
│   with aws_emr_studio.emr-studio,
│   on emr-studio.tf line 1, in resource "aws_emr_studio" "emr-studio":
│    1: resource "aws_emr_studio" "emr-studio" {

Studio code is:

resource "aws_emr_studio" "emr-studio" {
  name                        = "${local.full_name}-studio"
  auth_mode                   = "IAM"
  default_s3_location         = "s3://${var.s3_prevent_destroy == true ? aws_s3_bucket.emr[0].id : aws_s3_bucket.emr-destroy[0].id}/studio"
  engine_security_group_id    = aws_security_group.emr-studio.id
  service_role                = aws_iam_role.iam_emr_service_role.arn
  subnet_ids                  = [data.aws_subnets.intra.ids[0], data.aws_subnets.intra.ids[1], data.aws_subnets.intra.ids[2]]
  #user_role                   = aws_iam_role.iam_studio_user_role.arn
  vpc_id                      = data.aws_vpc.this.id
  workspace_security_group_id = aws_security_group.emr-studio.id
}

My iam role has

resource "aws_iam_role" "studio_service_role" {
  name = "${local.full_name}-studio-service-role"
  assume_role_policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
      {
        "Effect" : "Allow",
        "Principal" : {
          "Service" : "elasticmapreduce.amazonaws.com"
        },
        "Action" : "sts:AssumeRole",
    }]
  })
}

and the following policy attached:

resource "aws_iam_policy" "studio_service_role" {
  name        = "${local.full_name}-studio-service-policy"
  description = "copy of EMR_Notebooks_DefaultRole policy with extras"
  policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
       {
      "Sid": "AllowEMRReadOnlyActions",
      "Effect": "Allow",
      "Action": [
        "elasticmapreduce:ListInstances",
        "elasticmapreduce:DescribeCluster",
        "elasticmapreduce:ListSteps"
      ],
      "Resource": "*"
    },
    {
      "Sid": "AllowEC2ENIActionsWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateNetworkInterfacePermission",
        "ec2:DeleteNetworkInterface"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:network-interface/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
        }
      }
    },
    {
      "Sid": "AllowEC2ENIAttributeAction",
      "Effect": "Allow",
      "Action": [
        "ec2:ModifyNetworkInterfaceAttribute"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:instance/*",
        "arn:aws:ec2:*:*:network-interface/*",
        "arn:aws:ec2:*:*:security-group/*"
      ]
    },
    {
      "Sid": "AllowEC2SecurityGroupActionsWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:AuthorizeSecurityGroupEgress",
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:RevokeSecurityGroupEgress",
        "ec2:RevokeSecurityGroupIngress",
        "ec2:DeleteNetworkInterfacePermission"
      ],
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
        }
      }
    },
    {
      "Sid": "AllowDefaultEC2SecurityGroupsCreationWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateSecurityGroup"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:security-group/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
        }
      }
    },
    {
      "Sid": "AllowDefaultEC2SecurityGroupsCreationInVPCWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateSecurityGroup"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:vpc/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
        }
      }
    },
    {
      "Sid": "AllowAddingEMRTagsDuringDefaultSecurityGroupCreation",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateTags"
      ],
      "Resource": "arn:aws:ec2:*:*:security-group/*",
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true",
          "ec2:CreateAction": "CreateSecurityGroup"
        }
      }
    },
    {
      "Sid": "AllowEC2ENICreationWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateNetworkInterface"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:network-interface/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
        }
      }
    },
    {
      "Sid": "AllowEC2ENICreationInSubnetAndSecurityGroupWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateNetworkInterface"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:subnet/*",
        "arn:aws:ec2:*:*:security-group/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
        }
      }
    },
    {
      "Sid": "AllowAddingTagsDuringEC2ENICreation",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateTags"
      ],
      "Resource": "arn:aws:ec2:*:*:network-interface/*",
      "Condition": {
        "StringEquals": {
          "ec2:CreateAction": "CreateNetworkInterface"
        }
      }
    },
    {
      "Sid": "AllowEC2ReadOnlyActions",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeNetworkInterfaces",
        "ec2:DescribeTags",
        "ec2:DescribeInstances",
        "ec2:DescribeSubnets",
        "ec2:DescribeVpcs"
      ],
      "Resource": "*"
    },
    {
      "Sid": "AllowSecretsManagerReadOnlyActionsWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue"
      ],
      "Resource": "arn:aws:secretsmanager:*:*:secret:*",
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
        }
      }
    },
    {
      "Sid": "AllowWorkspaceCollaboration",
      "Effect": "Allow",
      "Action": [
        "iam:GetUser",
        "iam:GetRole",
        "iam:ListUsers",
        "iam:ListRoles",
        "sso:GetManagedApplicationInstance",
        "sso-directory:SearchUsers"
      ],
      "Resource": "*"
      },
      {
        "Effect" : "Allow",
        "Action" : "secretsmanager:GetSecretValue",
        "Resource" : "*"
      },
      {
        "Effect" : "Allow",
        "Action" : "s3:*",
        "Resource" : "*"
      },
      {
        "Effect" : "Allow",
        "Action" : [
          "ec2:AuthorizeSecurityGroupEgress",
          "ec2:AuthorizeSecurityGroupIngress",
          "ec2:CreateSecurityGroup",
          "ec2:DescribeSecurityGroups",
          "ec2:RevokeSecurityGroupEgress",
          "ec2:CreateNetworkInterface",
          "ec2:CreateNetworkInterfacePermission",
          "ec2:DeleteNetworkInterface",
          "ec2:DeleteNetworkInterfacePermission",
          "ec2:DescribeNetworkInterfaces",
          "ec2:ModifyNetworkInterfaceAttribute",
          "ec2:DescribeTags",
          "ec2:DescribeInstances",
          "ec2:DescribeSubnets",
          "ec2:DescribeVpcs",
          "elasticmapreduce:ListInstances",
          "elasticmapreduce:DescribeCluster",
          "elasticmapreduce:ListSteps"
        ],
        "Resource" : "*"
      },
      {
        "Effect" : "Allow",
        "Action" : "ec2:CreateTags",
        "Resource" : "arn:aws:ec2:*:*:network-interface/*",
        "Condition" : {
          "ForAllValues:StringEquals" : {
            "aws:TagKeys" : [
              "aws:elasticmapreduce:editor-id",
              "aws:elasticmapreduce:job-flow-id"
            ]
          }
        }
      }
    ]
  })
}

this is basically a copy and paste of all resource options from the following docs:

https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio-service-role.html

https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-notebooks-service-role.html

Any idea what im doing wrong?

已提问 1 年前1860 查看次数
5 回答
2

In your Terraform configuration, after creating the IAM role and policy, you also need to create an IAM role policy attachment. This resource links the policy to the role. Here's how you might do it:

resource "aws_iam_role_policy_attachment" "studio_service_role_attachment" {
  role       = aws_iam_role.studio_service_role.name
  policy_arn = aws_iam_policy.studio_service_role.arn
}

In your Terraform configuration, after creating the IAM role and policy, you also need to create an IAM role policy attachment. This resource links the policy to the role. Here's how you might do it:

resource "aws_iam_role_policy_attachment" "studio_service_role_attachment" {
  role       = aws_iam_role.studio_service_role.name
  policy_arn = aws_iam_policy.studio_service_role.arn
}
profile picture
专家
已回答 1 年前
0

Hi, @sdtslmn

Sorry yes i have attached the policy to the role, I also found that i have to remove all the tagging logic as I wont be adding tags to all the resources needed to match that pattern.

My iam-studio-service-role.tf now looks like below:

# More info for IAM can be found:
# https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio-service-role.html
# https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-notebooks-service-role.html

resource "aws_iam_role" "studio_service_role" {
  name = "${local.full_name}-studio-service-role"
  assume_role_policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
      {
        "Effect" : "Allow",
        "Principal" : {
          "Service" : "elasticmapreduce.amazonaws.com"
        },
        "Action" : "sts:AssumeRole",
    }]
  })
}

resource "aws_iam_role_policy_attachment" "studio_service_role" {
  role       = aws_iam_role.studio_service_role.name
  policy_arn = aws_iam_policy.studio_service_role.arn
}

resource "aws_iam_policy" "studio_service_role" {
  name        = "${local.full_name}-studio-service-policy"
  description = "copy of EMR_Notebooks_DefaultRole policy with extras"
  policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
       {
      "Sid": "AllowEMRReadOnlyActions",
      "Effect": "Allow",
      "Action": [
        "elasticmapreduce:ListInstances",
        "elasticmapreduce:DescribeCluster",
        "elasticmapreduce:ListSteps"
      ],
      "Resource": "*"
    },
    {
      "Sid": "AllowEC2ENIActionsWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateNetworkInterfacePermission",
        "ec2:DeleteNetworkInterface"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:network-interface/*"
      ],
    },
    {
      "Sid": "AllowEC2ENIAttributeAction",
      "Effect": "Allow",
      "Action": [
        "ec2:ModifyNetworkInterfaceAttribute"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:instance/*",
        "arn:aws:ec2:*:*:network-interface/*",
        "arn:aws:ec2:*:*:security-group/*"
      ]
    },
    {
      "Sid": "AllowEC2SecurityGroupActionsWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:AuthorizeSecurityGroupEgress",
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:RevokeSecurityGroupEgress",
        "ec2:RevokeSecurityGroupIngress",
        "ec2:DeleteNetworkInterfacePermission"
      ],
      "Resource": "*",
    },
    {
      "Sid": "AllowDefaultEC2SecurityGroupsCreationWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateSecurityGroup"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:security-group/*"
      ],
    },
    {
      "Sid": "AllowDefaultEC2SecurityGroupsCreationInVPCWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateSecurityGroup"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:vpc/*"
      ],
    },
    {
      "Sid": "AllowAddingEMRTagsDuringDefaultSecurityGroupCreation",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateTags"
      ],
      "Resource": "arn:aws:ec2:*:*:security-group/*",
    },
    {
      "Sid": "AllowEC2ENICreationWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateNetworkInterface"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:network-interface/*"
      ],
    },
    {
      "Sid": "AllowEC2ENICreationInSubnetAndSecurityGroupWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateNetworkInterface"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:subnet/*",
        "arn:aws:ec2:*:*:security-group/*"
      ],
    },
    {
      "Sid": "AllowAddingTagsDuringEC2ENICreation",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateTags"
      ],
      "Resource": "arn:aws:ec2:*:*:network-interface/*",
    },
    {
      "Sid": "AllowEC2ReadOnlyActions",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeNetworkInterfaces",
        "ec2:DescribeTags",
        "ec2:DescribeInstances",
        "ec2:DescribeSubnets",
        "ec2:DescribeVpcs"
      ],
      "Resource": "*"
    },
    {
      "Sid": "AllowSecretsManagerReadOnlyActionsWithEMRTags",
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue"
      ],
      "Resource": "arn:aws:secretsmanager:*:*:secret:*",
    },
    {
      "Sid": "AllowWorkspaceCollaboration",
      "Effect": "Allow",
      "Action": [
        "iam:GetUser",
        "iam:GetRole",
        "iam:ListUsers",
        "iam:ListRoles",
        "sso:GetManagedApplicationInstance",
        "sso-directory:SearchUsers"
      ],
      "Resource": "*"
      },
      {
        "Effect" : "Allow",
        "Action" : "secretsmanager:GetSecretValue",
        "Resource" : "*"
      },
      {
        "Effect" : "Allow",
        "Action" : "s3:*",
        "Resource" : "*"
      },
      {
        "Effect" : "Allow",
        "Action" : [
          "ec2:AuthorizeSecurityGroupEgress",
          "ec2:AuthorizeSecurityGroupIngress",
          "ec2:CreateSecurityGroup",
          "ec2:DescribeSecurityGroups",
          "ec2:RevokeSecurityGroupEgress",
          "ec2:CreateNetworkInterface",
          "ec2:CreateNetworkInterfacePermission",
          "ec2:DeleteNetworkInterface",
          "ec2:DeleteNetworkInterfacePermission",
          "ec2:DescribeNetworkInterfaces",
          "ec2:ModifyNetworkInterfaceAttribute",
          "ec2:DescribeTags",
          "ec2:DescribeInstances",
          "ec2:DescribeSubnets",
          "ec2:DescribeVpcs",
          "elasticmapreduce:ListInstances",
          "elasticmapreduce:DescribeCluster",
          "elasticmapreduce:ListSteps"
        ],
        "Resource" : "*"
      },
      {
        "Effect" : "Allow",
        "Action" : "ec2:CreateTags",
        "Resource" : "arn:aws:ec2:*:*:network-interface/*",
      }
    ]
  })
}

And i still get the error:

Error: creating EMR Studio: InvalidRequestException: The service role does not have permission to access the oc-dev-emr. Review the policies for the service role and the S3 bucket policy to ensure that the permission is granted. (Service: AWSEditors; Status Code: 400; Error Code: InvalidRequestException; Request ID: 38f2aa57-7ba0-48e0-899d-15c13db8c4f1; Proxy: null)
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "c8a17a22-609a-41b8-ac86-59950e786dbe"
│   },
│   Message_: "The service role does not have permission to access the oc-dev-emr. Review the policies for the service role and the S3 bucket policy to ensure that the permission is granted. (Service: AWSEditors; Status Code: 400; Error Code: InvalidRequestException; Request ID: 38f2aa57-7ba0-48e0-899d-15c13db8c4f1; Proxy: null)"
│ }
│ 
│   with aws_emr_studio.emr-studio,
│   on emr-studio.tf line 1, in resource "aws_emr_studio" "emr-studio":
│    1: resource "aws_emr_studio" "emr-studio" {
│ 

To add to my confusion! if i create a studio via the console using the same IAM role, it works successufully and lets bring up a workspace

已回答 1 年前
0

I am also having the same issues, any further update/resolution on this please? Thanks Suddhasil

已回答 9 个月前
0

for me below policy worked :

resource "aws_iam_policy" "emr_studio_policy" {
  name        = "emr_studio_policy"
  description = "Allow all accesses needed for EMR"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
          {
          "Effect": "Allow",
          "Action": [
            "elasticmapreduce:*",
            "ec2:CreateNetworkInterfacePermission",
            "ec2:DeleteNetworkInterface"
          ],
          "Resource": "*"
        },
        {
          "Effect": "Allow",
          "Action": "elasticmapreduce:StartNotebookExecution",
          "Resource": "arn:aws:elasticmapreduce:us-east-1:account_no:notebook-instance/*"
        },
        {
          "Effect": "Allow",
          "Action": [
            "s3:GetObject",
            "s3:PutObject"
          ],
          "Resource": [
            local.bucket_arn_with_wildcard
          ]
        },
        {
          "Effect": "Allow",
          "Action": "s3:ListBucket",
          "Resource": var.s3_bucket_arn
        }
    ]
  })
}

this is the policy code, and then had to attach this policy to the role created.

Chetan
已回答 8 个月前
0

This is my IAM role I succeed for EMR Studio. Hope it helps!

resource "aws_iam_role" "hm_amazon_emr_studio_iam_role" {
  name = "AmazonEMRStudioServiceRole-${var.amazon_emr_studio_name}"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = "sts:AssumeRole"
        Principal = {
          Service = "elasticmapreduce.amazonaws.com"
        }
      }
    ]
  })
  tags = {
    Environment = var.environment
    Team        = var.team
    Name        = "AmazonEMRStudioServiceRole-${var.amazon_emr_studio_name}"
  }
}
resource "aws_iam_role_policy" "hm_amazon_emr_studio_iam_role_input_policy" {
  name = "AmazonEMRStudioServicePolicyForS3-${var.amazon_emr_studio_name}"
  role = aws_iam_role.hm_amazon_emr_studio_iam_role.name
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "s3:DeleteObject",
          "s3:GetEncryptionConfiguration",
          "s3:GetObject",
          "s3:ListBucket",
          "s3:PutObject"
        ]
        Resource = [
          "arn:aws:s3:::${var.s3_bucket}",
          "arn:aws:s3:::${var.s3_bucket}/*"
        ]
      },
      {
        Effect = "Allow"
        Action = [
          "s3:ListAllMyBuckets"
        ]
        Resource = [
          "*"
        ]
      }
    ]
  })
}
resource "aws_iam_role_policy_attachment" "hm_amazon_emr_studio_iam_role_policy_attachment" {
  role       = aws_iam_role.hm_amazon_emr_studio_iam_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceEditorsRole"
}
profile picture
已回答 7 个月前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则

相关内容