- Newest
- Most votes
- Most comments
It appears that your MWAA environment is experiencing issues with reading remote logs from CloudWatch, which is preventing your DAGs from running properly. There are several potential reasons for this problem:
-
IAM Permissions: While your IAM policy includes permissions for CloudWatch Logs, there might be some missing or incorrect permissions. Ensure that the policy grants all necessary permissions for CloudWatch Logs operations.
-
Log Group Configuration: The error suggests that the specified log stream does not exist. This could be due to incorrect log group naming or issues with log stream creation.
-
Network Connectivity: For a public MWAA environment, ensure that the VPC has proper internet connectivity and that the security groups allow outbound traffic to CloudWatch Logs endpoints.
-
CloudWatch Agent Configuration: If you're using the CloudWatch agent, make sure it's correctly configured and running without errors.
To resolve this issue, try the following steps:
-
Double-check your IAM permissions. Ensure that the MWAA execution role has the correct permissions to access CloudWatch Logs, including creating log streams and putting log events.
-
Verify that the log group naming convention is correct. The log group should follow the format "airflow-<your-environment-name>-*".
-
Check the CloudWatch Logs console to see if the log group and log streams are being created correctly.
-
Ensure that your VPC has the necessary network configuration to allow communication with CloudWatch Logs endpoints.
-
If you're using a customer-managed KMS key for CloudWatch Logs encryption, make sure the MWAA execution role has the necessary permissions to use this key.
-
Consider increasing the logging level temporarily to get more detailed error information.
-
If the issue persists, you may need to review the MWAA environment's network configuration, including security groups and network ACLs, to ensure proper connectivity to AWS services.
If these steps don't resolve the issue, you may need to contact AWS support for further assistance, as there could be underlying configuration issues with your MWAA environment.
Sources
Resolve the read remote logs error in Amazon MWAA | AWS re:Post
Resolve empty log issues in CloudWatch | AWS re:Post
Troubleshoot permission errors for an instance on CloudWatch | AWS re:Post
Relevant content
- asked 2 years ago
- asked 2 years ago
- asked 2 years ago
- AWS OFFICIALUpdated 7 months ago
- AWS OFFICIALUpdated 4 months ago
- AWS OFFICIALUpdated 4 months ago
- AWS OFFICIALUpdated 6 months ago