Skip to content

How to setup multiple glue job for the same sns notification for cloudwatch alarm using CDK

0

I have to setup multiple gluejob for the same sns notification for cloudwatch alarm. Currently i am providing only one glue job name ''RTD table replication' in job failed rule but can i provide multiple glue job name in this rule ?

const jobFailedRule = new Rule(this, '{RTD table replication} JobFailureRule', {
      eventPattern: {
        source: ['aws.glue'],
        detailType: ['Glue Job State Change'],
        detail: {
          state: ['FAILED', 'TIMEOUT', 'ERROR'],
          jobName: ['RTD table replication'],
        },
      },
    })

  //cloudwatch metric
    const numFailedJobsMetric = new cloudwatch.Metric({
      namespace: 'AWS/Events',
      metricName: 'TriggeredRules',
      statistic: 'Sum',
      dimensionsMap: {
        RuleName: jobFailedRule.ruleName,
      },
    })
1 Answer
2
Accepted Answer

Yes, you can set up multiple Glue job names in the same CloudWatch alarm rule using CDK. To do this, you need to modify the jobName property in the detail section of your event pattern. Instead of providing a single job name, you can provide an array of job names.

Here's how you can modify your code to include multiple Glue job names:

const jobFailedRule = new Rule(this, 'MultipleJobsFailureRule', {
  eventPattern: {
    source: ['aws.glue'],
    detailType: ['Glue Job State Change'],
    detail: {
      state: ['FAILED', 'TIMEOUT', 'ERROR'],
      jobName: ['RTD table replication', 'Job2', 'Job3'], // Add multiple job names here
    },
  },
});

In this example, the rule will trigger for any of the specified job names ('RTD table replication', 'Job2', 'Job3') when they enter a FAILED, TIMEOUT, or ERROR state.

After setting up the rule with multiple job names, you can create a CloudWatch alarm based on the metric as you've done in your example:

const numFailedJobsMetric = new cloudwatch.Metric({
  namespace: 'AWS/Events',
  metricName: 'TriggeredRules',
  statistic: 'Sum',
  dimensionsMap: {
    RuleName: jobFailedRule.ruleName,
  },
});

This metric will now track failures across all the specified Glue jobs. You can then use this metric to create a CloudWatch alarm and link it to an SNS topic for notifications.

Remember to replace 'Job2' and 'Job3' with the actual names of your additional Glue jobs. This approach allows you to monitor multiple Glue jobs with a single rule and alarm, simplifying your monitoring setup.
Sources
@aws-cdk/aws-glue-alpha module · AWS CDK
Automating AWS Glue with CloudWatch Events - AWS Glue

answered a year ago
EXPERT
reviewed a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.