EKS Anywhere Observability using AWS native solution

Customer adopting EKS Anywhere for running and managing Kubernetes clusters on AWS cloud extended to on-premises, found it very difficult to setup observability as it was not possible using any AWS native solution and only 3rd party solution can be used to achieve this:

It was identified that this can be possible using AWS cloudWatch using Fluent Bit. Fluent Bit is an open source, multi-platform log processor and forwarder which allows you to collect data/logs from different sources, then unify and send them to multiple destinations. Due to its lightweight nature, using Fluent Bit as the log forwarder for EKS Anywhere clusters enables you to stream application logs into Amazon CloudWatch Logs efficiently and reliably.

Pre-requisite for setting up monitoring are:

1. An EKS Anywhere cluster with IAM Roles for Service Account (IRSA) enabled
2. Create an IAM Policy
3. Create an IAM Role
4. Create Service Account for both Fluent Bit and CloudWatch Agent
5. Deploy Fluent Bit in your EKS Anywhere cluster to scrape and send logs to CloudWatch

Customers can additionally use CloudWatch Container Insights to collect, aggregate, and summarise metrics and logs from your containerized applications and microservices running on EKS Anywhere clusters. CloudWatch automatically collects metrics for many resources, such as CPU, memory, disk, and network. Container Insights also provides diagnostic information, such as container restart failures, to help you isolate issues and resolve them quickly. You can also set CloudWatch alarms on metrics that Container Insights collects.

For more detail, please see below URLs:

1. https://anywhere.eks.amazonaws.com/docs/clustermgmt/observability/
2. https://github.com/aws/eks-anywhere/pull/6379

Co-authored by : Vibhuti Goel