By using AWS re:Post, you agree to the AWS re:Post Terms of Use

Cloudwatch config filter to exclude some JSON logs from being published to cloudwatch

0

I want to know how to write a filter expression that I can use in the amazon-cloudwatch-agent config to filter out JSON log entries that match a particular key-value. I am trying to add a cloudwatch config to my elasticbeanstalk which will exclude health check JSON logs from being published to Cloudwatch.

Jun 10 07:02:38 ip-x-y-p-q myapp[pid]: 
{
    "req": {
        "id": 7,
        "method": "GET",
        "url": "/health",
        "query": {},
        "params": {},
        ...
    },
}

Specifically I want to exclude logs with the key-value "url": "/health". My elasticbeanstalk config is like this:

files:
  "/opt/aws/amazon-cloudwatch-agent/bin/config.json":
    mode: "00600"
    owner: root
    group: root
    content: |
      {
        "agent": {
          "metrics_collection_interval": 10
        },
        "logs": {
          "logs_collected": {
            "files": {
              "collect_list": [
                {
                  "file_path": "/var/log/myapp.stdout.log",
                  "log_group_name": "`{ "Fn::Join": [ "/", [ "/aws/elasticbeanstalk", { "Ref": "AWSEBEnvironmentName" }, "var/log/myapp.log" ] ] }`",
                  "log_stream_name": "{instance_id}",
                  "retention_in_days": 7,
                  "filters": [
                     {
                        "type": "exclude",
                        "expression": "?"
                     }
                  ]
                }
              ]
            }
          }
        }
      }
container_commands:
  start_cloudwatch_agent:
    command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json

But my attempts seem to either exclude all logs so far. I have tried

  • ""url":\s"/health""
  • ""url": "/health""
  • "{ $.req.url = "/health"}"

In some cases it seems the cloudwatch agent itself doesn't start (presumably because of errors in the filter expression), when checked using sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status

1 Answer
0

Since the expression string contains special characters, try escaping those double quotes with backslashes so you have:

"expression": "\"url\": \"/health\""
AWS
answered 5 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions