Wait for ECS Run Task to Complete and Get Real Time Logs in Shell Script from GitHub Action

Question

I have a GitHub Action which includes a Shell Script Step. In this, I am executing AWS CLI command to run ECS task. Once I trigger this command, I want to read and print the CloudWatch logs in real time till the ECS run task is completed successfully. (Also, the other challenge is that the Cloud Watch Log Stream for the ECS Task Definition is named using some auto generated ID which cannot be controlled I think.) I am not sure how to achieve this, although I tried with the below code. I know there are some mistakes below, but couldn't find a good documentation to follow. Can anyone please help?

```
task_arn=$(aws ecs run-task --overrides file://${{ inputs.ecs_input_path }} --task-definition ${{ inputs.task_definition_arn }} --cluster ${{ inputs.cluster_arn }} --launch-type=${{ inputs.launch_type }} --network-configuration file://${{ inputs.network_config_path }} --query tasks[0].taskArn)
        echo $task_arn

timestamp=""
        while $(aws ecs wait tasks-stopped --cluster  --tasks "${task_arn}")
        do 
          aws logs tail ${log_group_name} --since ${timestamp} --output text
          timestamp=$(date +"%Y-%m-%dT%H:%M:%S%:z")
          sleep 5
        done
```

Answer

Hello,

I understand you want to read and print the CloudWatch logs in real time till the ECS run task is completed successfully.

To achieve real-time monitoring of CloudWatch logs during the execution of your ECS task, you can use the aws logs tail command in your shell script step. This command allows you to continuously stream logs from a CloudWatch log group.

Here's an example of how you can modify your script step:

aws logs tail "/ecs/your-log-group" --follow --filter-pattern "$TASK_ID" --format short

Explanation:

Run ECS Task: Execute your ECS task and capture the task ID from the output.

Determine Log Group: Identify the CloudWatch log group associated with your ECS task.

Tail Logs: Use the aws logs tail command to continuously stream logs from the specified log group. The --follow option ensures that new log events are continuously retrieved. The --filter-pattern option is used to filter logs based on the task ID or any other identifier relevant to your logs.

Remember to replace placeholders like your-log-group with your actual log group name.

Reference:

> https://awscli.amazonaws.com/v2/documentation/api/latest/reference/logs/tail.html

That being said, if you would like resource based troubleshooting, please raise a support case with AWS for further information and we will get back to you with on the support case.

Thank you!

Wait for ECS Run Task to Complete and Get Real Time Logs in Shell Script from GitHub Action

Relevant content