Secrets backend support in MWAA

Facing same issue, when I set Vault as secret backend MWAA env. stuck for couple of hours in create/update state and then fails after couple of hours

Using AWS Secrets manager works fine

Also, manually initiating VaultBackend class in Python operator to get secrets works secrets fine

So, I assume there is no connectivity issue in my case

It is very complicated to analyse such failure, as there is no log and Error message is pointing to documentation

Hello, As you may already aware, Amazon MWAA is a managed service for Apache Airflow that lets you use your current, familiar Apache Airflow platform to orchestrate your workflows. MWAA runs open source Apache Airflow. So if Airflow supports a third party integration, then usually you can be able to use it with MWAA.

Airflow supports Hashicorp Vault as shown here. https://airflow.apache.org/docs/apache-airflow-providers-hashicorp/stable/secrets-backends/hashicorp-vault.html. The possible reasons for this failure could be IAM permissions issues or networking issues. If the vault is stored externally then please ensure whether you are able to connect it from the subnet you are using in MWAA. Please make sure to have a route to connect to vault if is placed inside EC2.

Usage of a custom secrets backend is outside of the scope of the MWAA service team, and in general we cannot provide specific guidance. Note that the secrets backend may need to be accessible from the web server, depending on how it's implemented, and web servers in MWAA cannot access VPCs outside AWS.