How do I troubleshoot my Amazon MWAA environment that's stuck in the "Creating" state?

4 minute read
0

I tried to create an Amazon Managed Workflows for Apache Airflow (Amazon MWAA) environment, but it's stuck in the "Creating" state.

Short description

To troubleshoot your Amazon MWAA environment, complete the following steps:

  1. Run the AWSSupport-TroubleshootMWAAEnvironmentCreation runbook.
  2. Based on the output, complete the Resolution steps in the related section.

Resolution

Prerequisite

Make sure that your AWS Identify and Access Management (IAM) user or role has the required permissions. For more information, see the Required IAM permissions section of AWSSupport-TroubleshootMWAAEnvironmentCreation.

Run the AWSSupport-TroubleshootMWAAEnvironmentCreation runbook

  1. Open the AWS Systems Manager console.
  2. In the navigation pane, choose Automation, and then choose Execute automation.
  3. On the Choose runbook page, in the Automation runbook field, enter AWSSupport-TroubleshootMWAA. Then, select AWSSupport-TroubleshootMWAA.
  4. On the AWSSupport-TroubleshootMWAAEnvironmentCreation page, choose Execute automation.
  5. On the Execute automation runbook page, select the following:
    For EnvironmentName, enter the name of the MWAA environment that you want to troubleshoot.
    (Optional) For AutomationAssumeRole, enter the ARN of the IAM role that performs actions on your behalf.
  6. Choose Execute.

If your environment is stuck for more than 30 minutes in the "Creating" state, then the issue might involve your networking configuration. The root cause of the issue and the appropriate resolution depend on your networking setup.

Your network configuration lacks the route to AWS services or the internet

To resolve this issue, based on the type of routing you choose, verify that the network configuration meets the prerequisites for the environment:

  • Public routing: Your Amazon Virtual Private Cloud (Amazon VPC) infrastructure must have two public and two private subnets. Public subnets get public IP addresses and have the default route to the internet gateway. Private subnets get only private IP addresses and have no route to the internet gateway. Instead, private subnets have only a route to the NAT gateway. For more information, see Public routing over the internet. Typically, the network flow with public routing looks similar to the following:
    Private subnet - Default route to NAT gateway - NAT gateway associated with the public subnet - public subnet - default route to the internet gateway - internet
  • Private routing: To use the Apache Airflow on MWAA, your Amazon VPC that doesn't have internet access must have additional VPC service endpoints. These Amazon VPC endpoints include Amazon S3, monitoring, ecr.dkr, ecr.api, logs, sqs, kms, airflow.api, airflow.env, and airflow.ops. For more information, see Creating the required VPC service endpoints in an Amazon VPC with private routing and Private routing without internet access. The VPC endpoints must have private DNS turned on. Verify that the endpoints are associated with the environment's subnets and security group. Also, configure the VPC endpoint policy for each endpoint to allow full access to the endpoint.

The security group or network access control list (ACL) restricts the network traffic

To resolve this issue, verify that the security group specifies a self-referencing inbound rule to itself or the port range HTTPS 443 and TCP 5432. The security group must specify an outbound rule for all traffic. The network ACL must have an inbound or outbound rule that allows all traffic. For an example, see Example ACLs.

Downloading the container image from Amazon ECR has failed

If you use an Amazon VPC without internet access, create an Amazon S3 gateway endpoint. Then, grant the minimum required permissions to Amazon ECR to access Amazon S3 in that AWS Region.

To troubleshoot issues related to the Amazon VPC network routing, see I tried to create an environment and it's stuck in the "Creating" state.

Related information

About networking on Amazon MWAA

AWS Support Automation Workflows (SAW)

Running a simple automation

Setting up Automation

AWS OFFICIAL
AWS OFFICIALUpdated 9 months ago
2 Comments

This article misses an important point: You can't take any of these steps, because an MWAA environment stuck in 'Creating' doesn't allow for any changes, nor can you delete the environment.

"Environments with CREATING status must complete previous operation before initiating a new operation."

replied a year ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

profile pictureAWS
EXPERT
replied a year ago