Secrets backend support in MWAA

2

Is any other secrets backend besides AWS Secrets manager supported in MWAA? I would like to use Hashicorp Vault, and the Airflow provider loads fine, but when i try to set secrets.backend_kwargs it seems fail.

BenO
已提問 1 年前檢視次數 687 次
2 個答案
1

Facing same issue, when I set Vault as secret backend MWAA env. stuck for couple of hours in create/update state and then fails after couple of hours

Using AWS Secrets manager works fine

Also, manually initiating VaultBackend class in Python operator to get secrets works secrets fine

So, I assume there is no connectivity issue in my case

It is very complicated to analyse such failure, as there is no log and Error message is pointing to documentation

Grisha
已回答 1 年前
-1

Hello, As you may already aware, Amazon MWAA is a managed service for Apache Airflow that lets you use your current, familiar Apache Airflow platform to orchestrate your workflows. MWAA runs open source Apache Airflow. So if Airflow supports a third party integration, then usually you can be able to use it with MWAA.

Airflow supports Hashicorp Vault as shown here. https://airflow.apache.org/docs/apache-airflow-providers-hashicorp/stable/secrets-backends/hashicorp-vault.html. The possible reasons for this failure could be IAM permissions issues or networking issues. If the vault is stored externally then please ensure whether you are able to connect it from the subnet you are using in MWAA. Please make sure to have a route to connect to vault if is placed inside EC2.

Usage of a custom secrets backend is outside of the scope of the MWAA service team, and in general we cannot provide specific guidance. Note that the secrets backend may need to be accessible from the web server, depending on how it's implemented, and web servers in MWAA cannot access VPCs outside AWS.

AWS
已回答 1 年前
  • One follow on question, when the MWAA is in Private network mode, is it possible for the web servers to initiate traffic back into my VPC? Our vault is in our VPC.

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南