Nagios and Cloudwatch integration

0

A customer wants to explore the possibility to send some Cloudwatch alerts based on certain conditions, to their existing Nagios Dashboard.

Have found that there are some plugins that could be a valid solution: https://github.com/level23/nagios-cloudwatch-metrics

Has anybody an up to date workaround that may fit or previous customer experiences?

asked 6 years ago774 views
1 Answer
0
Accepted Answer

I handled nagios in production around 6 years ago, so my information may be outdated. Consider this as a starting point for a potential 'hack' of a solution:

  • Cloudwatch -> SNS (API call: https://SomeEndpoint ) EX: php/python script served by nginx/apache with basic/digest auth

  • On alarm; the above endpoint call a function/script that changes nagios alert depending on incoming alert. NAGIOS can mimic cloudwatch's status by leveraging its own state: OK, WARNING, UP, DOWN etc.. so script will parse incoming content within the API call and act accordingly.

Possible advantages:

  1. Reduce hits on API, as NAGIOS will not constantly hit cloudwatch API. it's being triggered only on cloudwatch status change.
  2. No credentials needed on Nagios box or any other box for that matter, as this is just a restful API call initiated by SNS
  3. SNS api call will handle the retry for you in case of restful api hiccup.
  4. Optional: As a fail-safe, to make sure you are alerted if/when an alarm does not reach NAGIOS. you can do the following: 4.1. In addition to the mentioned API call; you can fan out the SNS topic to an SQS queue with a cloudwatch monitor of >1 message for longer than 5 minutes.
    At the end of our endpoint triggered script (mentioned above) you can add a function that deletes related message from queue. if script did not execute. (aka failed) message will remain in queue and CloudWatch SQS alarm will go off and do whatever you need it to do, such as send e-mail to OPS mailing list.
answered 6 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions