How to use cloud watch agent to upload only logs of Ubuntu user login failures to cloudwatch

Question

In the var/log/auth.log authentication log file, login failure information usually begins with keywords such as "password failure" and "authentication failure". How to configure the proxy file /var/awslogs/etc/awslogs.conf to only collect user login failure information and send it to cloudwatch

ChatGPT gave a solution, but it didn’t work. It uploaded the entire auth.log content.

```
[/var/log/auth.log]
datetime_format = %b %d %H:%M:%S
file = /var/log/auth.log
buffer_duration = 5000
log_stream_name = {instance_id}
initial_position = start_of_file
log_group_name = your-log-group-name

filter_pattern = [timestamp, loglevel, source, message, host, program, pid] !~ ".*sshd.*Failed (password|publickey).*"
```

Accepted Answer

Hello.

I think the file you are looking at is probably the configuration file for CloudWatch Logs Agent.  
CloudWatch Logs Agent is an older agent, so you should normally use CloudWatch Agent.  
If you are using CloudWatch Logs Agent, there is no "filter_pattern" setting, so I don't think you can filter.  
https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AgentReference.html

If you are using CloudWatch Agent, there is a setting called "filters", so you can filter the logs you want to send.  
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html#CloudWatch-Agent-Configuration-File-Logssection  
```
"collect_list": [ 
  {
    "file_path": "/opt/aws/amazon-cloudwatch-agent/logs/test.log", 
    "log_group_name": "test.log", 
    "log_stream_name": "test.log",
    "filters": [
      {
        "type": "exclude",
        "expression": "Firefox"
      },
      {
        "type": "include",
        "expression": "P(UT|OST)"
      }
    ]
  },
  .....
]
```

How to use cloud watch agent to upload only logs of Ubuntu user login failures to cloudwatch

Relevant content