关于CUR生成的crawler-cfn.yml模板在cloudformation中运行在AWSCURCrawler创建报错denied access(CUR Crawler Resource Fails with Access Denied in CFN Stack)

Question

求助各位，我在尝试用grafana以payer账号为数据源去监控成本，为iam账号管理员，在成本和使用情况报告中创建CUR报告，用CUR报告第一次输出到s3桶的crawler-cfn.yml模板去cloudformation中运行，在逻辑ID为AWSCURCrawler事件中报错Resource handler returned message: "Account  is denied access. (Service: Glue, Status Code: 400, Request ID: 4214d733-aa8b-42d4-9803-410a12f1726b)" (RequestToken: ](), HandlerErrorCode: AccessDenied),我在cloudtrail那边搜索了关于CreateCrawler的事件，查看报错如下：
    "eventTime": "2025-06-27T01:25:23Z",
    "eventSource": "glue.amazonaws.com",
    "eventName": "CreateCrawler",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "cloudformation.amazonaws.com",
    "userAgent": "cloudformation.amazonaws.com",
    "errorCode": "AccessDenied",
    "errorMessage": "Account  is denied access.",
    "requestParameters": {
        "role": "arn:aws:iam:::role/billing-public-grafana-cf-AWSCURCrawlerComponentFun-kNp83d1iAW4b",
        "databaseName": "athenacurcfn_cost_display_monitor",
        "schemaChangePolicy": {
            "updateBehavior": "UPDATE_IN_DATABASE",
            "deleteBehavior": "DELETE_FROM_DATABASE"
        }
这些资源区域都在us-east-1.同一账号，s3桶为创建CUR报告那个页面创建的新桶附加默认的策略外未附加别的策略。
事实上，我已经在其他payer账号上做了很多次相同的操作都是成功的，只有这个失败了，账号权限一致。以下为脱敏版本的cf模板：
AWSTemplateFormatVersion: 2010-09-09
Resources:

YourGlueDatabase:
    Type: 'AWS::Glue::Database'
    Properties:
      DatabaseInput:
        Name: 'your-database-name'
      CatalogId: !Ref AWS::AccountId

YourCrawlerRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - glue.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      ManagedPolicyArns:
        - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSGlueServiceRole'
      Policies:
        - PolicyName: YourCrawlerPolicy
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'logs:CreateLogGroup'
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvents'
                Resource: !Sub 'arn:${AWS::Partition}:logs:*:*:*'
              - Effect: Allow
                Action:
                  - 'glue:UpdateDatabase'
                  - 'glue:UpdatePartition'
                  - 'glue:CreateTable'
                  - 'glue:UpdateTable'
                  - 'glue:ImportCatalogToGlue'
                Resource: '*'
              - Effect: Allow
                Action:
                  - 's3:GetObject'
                  - 's3:PutObject'
                Resource: !Sub 'arn:${AWS::Partition}:s3:::example-bucket/your-path/your-report*'
        - PolicyName: YourKMSDecryption
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'kms:Decrypt'
                Resource: '*'

YourLambdaRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      Policies:
        - PolicyName: YourLambdaExecutorPolicy
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'logs:CreateLogGroup'
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvents'
                Resource: !Sub 'arn:${AWS::Partition}:logs:*:*:*'
              - Effect: Allow
                Action:
                  - 'glue:StartCrawler'
                Resource: '*'

YourGlueCrawler:
    Type: 'AWS::Glue::Crawler'
    DependsOn:
      - YourGlueDatabase
      - YourCrawlerRole
    Properties:
      Name: your-crawler-name
      Description: Keeps your CUR table in Athena up-to-date.
      Role: !GetAtt YourCrawlerRole.Arn
      DatabaseName: !Ref YourGlueDatabase
      Targets:
        S3Targets:
          - Path: 's3://example-bucket/your-path/your-report'
            Exclusions:
              - '**.json'
              - '**.yml'
              - '**.sql'
              - '**.csv'
              - '**.gz'
              - '**.zip'
      SchemaChangePolicy:
        UpdateBehavior: UPDATE_IN_DATABASE
        DeleteBehavior: DELETE_FROM_DATABASE

YourInitializerFunction:
    Type: 'AWS::Lambda::Function'
    DependsOn: YourGlueCrawler
    Properties:
      Code:
        ZipFile: >
          const { GlueClient, StartCrawlerCommand } = require('@aws-sdk/client-glue');
          const response = require('./cfn-response');
          exports.handler = function (event, context, callback) {
            if (event.RequestType === 'Delete') {
              response.send(event, context, response.SUCCESS);
            } else {
              const glue = new GlueClient();
              const input = { Name: 'your-crawler-name' };
              const command = new StartCrawlerCommand(input);
              glue.send(command, function (err, data) {
                if (err) {
                  const responseData = JSON.parse(this.httpResponse.body);
                  if (responseData['__type'] == 'CrawlerRunningException') {
                    callback(null, responseData.Message);
                  } else {
                    const responseString = JSON.stringify(responseData);
                    if (event.ResponseURL) {
                      response.send(event, context, response.FAILED, { msg: responseString });
                    } else {
                      callback(responseString);
                    }
                  }
                } else {
                  if (event.ResponseURL) {
                    response.send(event, context, response.SUCCESS);
                  } else {
                    callback(null, response.SUCCESS);
                  }
                }
              });
            }
          };
      Handler: 'index.handler'
      Timeout: 30
      Runtime: nodejs18.x
      ReservedConcurrentExecutions: 1
      Role: !GetAtt YourLambdaRole.Arn

StartCrawlerCustomResource:
    Type: 'Custom::StartCrawler'
    Properties:
      ServiceToken: !GetAtt YourInitializerFunction.Arn

S3LambdaPermission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: 'lambda:InvokeFunction'
      FunctionName: !GetAtt YourInitializerFunction.Arn
      Principal: 's3.amazonaws.com'
      SourceAccount: !Ref AWS::AccountId
      SourceArn: !Sub 'arn:${AWS::Partition}:s3:::example-bucket'

S3LambdaNotificationRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      Policies:
        - PolicyName: S3NotificationPolicy
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'logs:CreateLogGroup'
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvents'
                Resource: !Sub 'arn:${AWS::Partition}:logs:*:*:*'
              - Effect: Allow
                Action:
                  - 's3:PutBucketNotification'
                Resource: !Sub 'arn:${AWS::Partition}:s3:::example-bucket'

ConfigureS3NotificationFunction:
    Type: 'AWS::Lambda::Function'
    DependsOn:
      - YourInitializerFunction
      - S3LambdaPermission
      - S3LambdaNotificationRole
    Properties:
      Code:
        ZipFile: >
          const { S3Client, PutBucketNotificationConfigurationCommand } = require('@aws-sdk/client-s3');
          const response = require('./cfn-response');
          exports.handler = function (event, context, callback) {
            const s3 = new S3Client();
            const putConfigRequest = function (notificationConfiguration) {
              return new Promise(function (resolve, reject) {
                const input = {
                  Bucket: event.ResourceProperties.BucketName,
                  NotificationConfiguration: notificationConfiguration,
                };
                const command = new PutBucketNotificationConfigurationCommand(input);
                s3.send(command, function (err, data) {
                  if (err) reject({ msg: this.httpResponse.body.toString(), error: err, data: data });
                  else resolve(data);
                });
              });
            };
            const newNotificationConfig = {};
            if (event.RequestType !== 'Delete') {
              newNotificationConfig.LambdaFunctionConfigurations = [
                {
                  Events: ['s3:ObjectCreated:*'],
                  LambdaFunctionArn: event.ResourceProperties.TargetLambdaArn || 'missing arn',
                  Filter: {
                    Key: {
                      FilterRules: [
                        { Name: 'prefix', Value: event.ResourceProperties.ReportKey },
                      ],
                    },
                  },
                },
              ];
            }
            putConfigRequest(newNotificationConfig)
              .then(function (result) {
                response.send(event, context, response.SUCCESS, result);
                callback(null, result);
              })
              .catch(function (error) {
                response.send(event, context, response.FAILED, error);
                console.log(error);
                callback(error);
              });
          };
      Handler: 'index.handler'
      Timeout: 30
      Runtime: nodejs18.x
      ReservedConcurrentExecutions: 1
      Role: !GetAtt S3LambdaNotificationRole.Arn

CreateS3Notification:
    Type: 'Custom::CreateS3Notification'
    Properties:
      ServiceToken: !GetAtt ConfigureS3NotificationFunction.Arn
      TargetLambdaArn: !GetAtt YourInitializerFunction.Arn
      BucketName: 'example-bucket'
      ReportKey: 'your-path/your-report'

CURStatusGlueTable:
    Type: 'AWS::Glue::Table'
    DependsOn: YourGlueDatabase
    Properties:
      DatabaseName: your-database-name
      CatalogId: !Ref AWS::AccountId
      TableInput:
        Name: 'cost_and_usage_data_status'
        TableType: 'EXTERNAL_TABLE'
        StorageDescriptor:
          Columns:
            - Name: status
              Type: 'string'
          InputFormat: 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'
          OutputFormat: 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
          SerdeInfo:
            SerializationLibrary: 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
          Location: 's3://example-bucket/your-path/cost_and_usage_data_status/'
参考链接：https://aws.amazon.com/cn/blogs/china/visualizing-cloud-resource-costs-and-usage-based-on-grafana/#:~:text=%E5%9C%A8%E6%9C%AC%E7%AF%87%E5%8D%9A%E5%AE%A2%E4%B8%AD%EF%BC%8C%E6%88%91%E4%BB%AC%E5%B0%86%E6%BC%94%E7%A4%BA%E5%A6%82%E4%BD%95%E4%BD%BF%E7%94%A8%20CUR%EF%BC%8C%E7%BB%93%E5%90%88%20Amazon%20Athena,%E5%92%8C%E5%BC%80%E6%BA%90%E7%89%88%20Grafana%20%E6%90%AD%E5%BB%BA%E5%AE%9A%E5%88%B6%E5%8C%96%E7%9A%84%E4%BB%AA%E8%A1%A8%E7%9B%98%EF%BC%8C%E5%8F%AF%E8%A7%86%E5%8C%96%E5%9C%B0%E5%B1%95%E7%A4%BA%E6%88%90%E6%9C%AC%E6%94%AF%E5%87%BA%E3%80%82%20%E5%9F%BA%E4%BA%8E%20CUR%EF%BC%8C%E5%8F%AF%E4%BB%A5%E5%AE%9A%E5%88%B6%E5%8C%96%E4%B8%8D%E5%90%8C%E7%9A%84%E4%BB%AA%E8%A1%A8%E7%9B%98%E6%9D%A5%E4%B8%BA%E5%AE%A2%E6%88%B7%E6%8F%90%E4%BE%9B%E4%B8%8D%E5%90%8C%E8%A7%86%E8%A7%92%E7%9A%84%E6%88%90%E6%9C%AC%E5%8F%AF%E8%A7%86%E5%8C%96%E4%BF%A1%E6%81%AF%E3%80%82

re:Post Agent · Accepted Answer

When encountering an "Access Denied" error while creating a Glue crawler through CloudFormation using the CUR-generated crawler-cfn.yml template, this is typically an account-level issue rather than a permissions problem with your IAM role.

Based on the error message "Account is denied access. (Service: Glue, Status Code: 400)" and the CloudTrail logs showing the same error, this appears to be a limitation on your AWS account rather than an issue with your template or IAM permissions.

This is a known issue that can occur particularly with newer AWS accounts. Even though you have successfully performed the same operation on other payer accounts, this specific account may have restrictions that prevent it from creating Glue crawlers.

The solution is to open a support ticket with AWS Support under the "Account and billing" category. This type of support request can be made free of charge. In your ticket, explain that you're encountering an "Access Denied" error when trying to create a Glue crawler, despite having the appropriate permissions. The AWS team will need to check your account and grant it the necessary permissions to create crawlers.

Several users have reported similar issues and had them resolved by contacting AWS Support, as the AWS team needed to manually enable or verify the account's ability to create crawlers, especially for newer accounts.  
**Sources**  
[About Crawler creation error | AWS re:Post](https://repost.aws/questions/QULTBGFXejRzaPgsDizguNBg/about-crawler-creation-error)  
[Access Denied Error While Creating Crawler | AWS re:Post](https://repost.aws/questions/QU2mnSyZXSQWuHrJ7bGgxsAQ/access-denied-error-while-creating-crawler)

关于CUR生成的crawler-cfn.yml模板在cloudformation中运行在AWSCURCrawler创建报错denied access(CUR Crawler Resource Fails with Access Denied in CFN Stack)

Add your answer

Relevant content