Exclude nginx logs from Cloudwatch log group

Hi,

Elastic Beanstalk uses the awslogs agent to stream logs to CloudWatch. If you want to customize which logs are streamed, you'll need to modify the awslogs configuration.

Here are the steps to achieve this:

1. Understand the Default Configuration: Elastic Beanstalk's default setup has a set of predefined logs it streams to CloudWatch. These configurations are usually stored in the /etc/awslogs/config/ directory. There might be a file named elasticbeanstalk.conf or similar that contains the log stream configurations.

2. Create a Custom Configuration in .ebextensions: You can override the default configuration using .ebextensions. Here's a sample configuration that only streams web.stdout.log:

files:
  "/etc/awslogs/config/web-stdout.conf":
    mode: "000644"
    owner: root
    group: root
    content: |
      [/var/log/nginx/web.stdout.log]
      datetime_format = %Y-%m-%dT%H:%M:%S.%fZ
      file = /var/log/nginx/web.stdout.log
      buffer_duration = 5000
      log_stream_name = {instance_id}
      initial_position = start_of_file
      log_group_name = your-log-group-name

Replace your-log-group-name with the desired CloudWatch log group name.

3. Remove Default Configuration: You can also add commands in your .ebextensions to delete the default configuration files to ensure they don't interfere:

commands:
  01_remove_default_log_configs:
    command: "rm -f /etc/awslogs/config/*.conf"

4. Restart the awslogs Agent: After modifying the configuration, you should restart the awslogs agent to pick up the changes. You can add this to your .ebextensions:

5. Combine Everything: Combine all of the above configurations into a single .config file in the .ebextensions directory of your Elastic Beanstalk application source bundle.

files:
  "/etc/awslogs/config/web-stdout.conf":
    mode: "000644"
    owner: root
    group: root
    content: |
      [/var/log/nginx/web.stdout.log]
      datetime_format = %Y-%m-%dT%H:%M:%S.%fZ
      file = /var/log/nginx/web.stdout.log
      buffer_duration = 5000
      log_stream_name = {instance_id}
      initial_position = start_of_file
      log_group_name = your-log-group-name

commands:
  01_remove_default_log_configs:
    command: "rm -f /etc/awslogs/config/*.conf"
  02_restart_awslogs:
    command: "service awslogs restart"

6. Deploy the Updated Configuration: Once you've added the .config file to your .ebextensions directory, re-deploy your Elastic Beanstalk application. This will apply the changes and only web.stdout.log should be streamed to CloudWatch.

Remember to monitor your application closely after making these changes to ensure everything is working as expected.