By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

MWAA retrieving aws_default connection from Secrets Manager millions of times per month

1

We have a small environment (~30 DAGs) running Airflow 2.2.2 with Secrets Manager as our secrets backend. We noticed once we switched to Secrets Manager that MWAA was retrieving the aws_default connection from Secrets Manager multiple times per second.

After digging into the logs, it seems to be the scheduler making all the calls but I'm not sure if that's normal behavior or what needs to change in our configuration to get the environment to search elsewhere for the execution role credentials.

  • Keegan
    10 months ago

    Were you able to every find a solution to this issue? I would like to not spend $20 a month on bogus api calls.

Topics
Application IntegrationSecurity, Identity, & Compliance
Tags
Amazon Managed Workflows for Apache Airflow (MWAA)AWS Secrets Manager
Language
English
ttw2022
asked a year ago912 views
2 Answers
0

You can control the Secrets Manager API call by MWAA by having "connections_lookup_pattern" with "secrets.backend_kwargs" configuration.

Configuring an Apache Airflow connection using a AWS Secrets Manager secret - Step two: Create the Secrets Manager backend as an Apache Airflow configuration option - https://docs.aws.amazon.com/mwaa/latest/userguide/connections-secrets-manager.html#connections-sm-aa-configuration

https://airflow.apache.org/docs/apache-airflow-providers-amazon/7.3.0/_api/airflow/providers/amazon/aws/secrets/secrets_manager/index.html

Note if you are using the 7.3 provider version and you are on private you will need to update it in your plugins.zip. Do not follow instructions that tell you to put a constraints.txt in you dags folder it will not upload and will crash your EVN.

Keegan
answered 10 months ago
-1

Every time a boto session is created many log lines are added regarding how the session was constructed. Looks like change has been pushed to change the logging to debug . please find
https://github.com/apache/airflow/pull/20858

Priyatham
answered a year ago
  • ttw2022
    a year ago

    I saw that, but my issue isn't necessarily with it spamming the logs, but making millions of calls to Secrets Manager per month. I'd like to be able to point it somewhere else when creating a boto session (especially since it's not actually using the credentials from Secrets Manager to do create that Session).

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content