How to get cloudwatch agent working on ECS with terraform

0

I'm doing my best to follow these instructions. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Generation_CloudWatch_Agent.html

I've added a container definition to my terraform

resource "aws_ecs_task_definition" "taskdef" {
    family = "my_service${var.suffix}_service"
    container_definitions = jsonencode([
        {
            assign_public_ip = true
            name = "cloudwatch-agent"
            image = "public.ecr.aws/cloudwatch-agent/cloudwatch-agent:latest"
            memory = 256
            cpu = 256
            portMappings = [
                {
                    protocol = "tcp"
                    containerPort = 25888
                },
                {
                    protocol = "udp"
                    containerPort = 25888
                }
            ]
            logConfiguration = {
                logDriver = "awslogs"
                options = {
                    awslogs-create-group = "true"
                    awslogs-group = "/aws/ecs/my-log-group"
                    awslogs-region = "us-west-2"
                    awslogs-stream-prefix = "ecs"
                }
            }
            Environment = [
                {"name": "CW_CONFIG_CONTENT", "value": "{\"logs\": { \"metrics_collected\": { \"emf\": { }}}}"}
            ]
        },
        {
            assign_public_ip = true
            name = "my-service${var.suffix}container"
            cpu = 1024
            memory = 2048
            image = "${var.image-name}@${var.image-hash}"
            portMappings = [
                {
                    name = "http"
                    containerPort = 80
                    hostPort = 80
                    protocol = "http"
                }
            ]
            logConfiguration = {
                logDriver = "awslogs"
                options = {
                    awslogs-create-group = "true"
                    awslogs-group = "/aws/ecs/my-log-group"
                    awslogs-region = "us-west-2"
                    awslogs-stream-prefix = "ecs"
                }
            }
            health_check = {

            }
            Environment = [
                {"name": "CLOUDWATCH_LOG_GROUP", "value": aws_cloudwatch_log_group.metrics.name},   
                {"name": "AWS_EMF_AGENT_ENDPOINT", "value": "tcp://127.0.0.1:25888"},
                {"name": "METRICS_NAMESPACE", "value": "athena-scanrunner${var.suffix}-metrics"}
            ]
        }
    ])
    cpu = 2048
    execution_role_arn = var.service_role
    task_role_arn = var.service_role
    memory = 4096
    network_mode = "awsvpc"
    requires_compatibilities = ["FARGATE"]
    runtime_platform {
        cpu_architecture = "ARM64"
        operating_system_family = "LINUX"
    }
}

I couldn't get the valueFrom field to work so I used JSON directly for the config.

In my code I've added the config

Amazon.CloudWatch.EMF.Config.EnvironmentConfigurationProvider.Config =
        new Amazon.CloudWatch.EMF.Config.Configuration
        {
            ServiceName = "Athena-ScanRunner",
            ServiceType = "WebApi",
            LogGroupName = Environment.GetEnvironmentVariable("CLOUDWATCH_LOG_GROUP"),
            AgentEndPoint = "tcp://127.0.0.1:25888",
            EnvironmentOverride = Amazon.CloudWatch.EMF.Environment.Environments.ECS
        };


    var builder = WebApplication.CreateBuilder(args);
    builder.Configuration.GetAWSOptions();
    builder.Services.AddControllers().AddJsonOptions(options =>
    {
        options.JsonSerializerOptions.Converters.Add(new JsonStringEnumConverter());

        options.JsonSerializerOptions.DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull;
    });
    builder.Services.AddHealthChecks();
    builder.Services.AddAWSService<IAmazonDynamoDB>();
    builder.Services.AddAWSService<IAmazonECS>();
    builder.Services.AddAWSService<IAmazonS3>();
    builder.Services.AddAWSService<IAmazonSQS>();
    builder.Services.AddEmf();
    builder.Services.AddSingleton<IScanTaskDb, ScanTaskDynamoDb>();
    builder.Services.AddSingleton<ITaskQueue, SqsTaskQueue>();
    builder.Services.AddSingleton<IFileStore, S3FileStore>();


    var app = builder.Build();

    app.UseHealthChecks("/");
    app.UseEmfMiddleware();

and I've added the metrics to each api endpoint.

I get no metrics flowing. How do I make this setup work?

1 Respuesta
1

Unfortunately you haven’t shared the full VPC setup or task role definition. However.

Do you have the ecs task running in a public subnet?

Does the task have the correct IAM permissions to create log groups and streams?

Also you should place your CW Agent Config in an SSM parameter and let the container read it in as an environment variable.

profile picture
EXPERTO
respondido hace 2 meses
profile picture
EXPERTO
revisado hace 2 meses
  • Yeah public subnet, you can also see the assigned public IP.

    Permissions look correct, I'm getting no errors and normal logs work correctly.

    Also you should place your CW Agent Config in an SSM parameter and let the container read it in as an environment variable.

    valueFrom doesn't work with terraform or entering the JSON in the task definition editor. I can't do this if the mechanism specified in the docs just silently fails.

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas