How do I use Amazon SES as the SMTP host to send emails from Amazon MWAA DAG tasks?

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

To use Amazon SES as the SMTP host to send emails from a DAG task on Amazon MWAA, complete the following steps:

1. Set up email sending with Amazon SES.
2. Create Amazon SES SMTP credentials to send emails in Amazon MWAA.
   Note: The SMTP interface credentials are different from the access keys that you create with AWS Identity Access and Management (IAM) for an SMTP user.
3. Attach your Apache Airflow configuration options to your Amazon MWAA environment.
4. Set the following values for the configuration options:
   Set email.email_backend to airflow.utils.email.send_email_smtp.
   Set smtp.smtp_host to email-smtp.region.amazonaws.com.
   Note: Replace region with your AWS Region.
   Set smtp.smtp_starttls to False.
   Set smtp.smtp_ssl to True.
   Set smtp.smtp_port to 587.
   Note: Use port 587 for SMTP traffic. By default, AWS blocks outbound SMTP traffic on port 25 from all Amazon Elastic Compute Cloud (Amazon EC2) instances. To send outbound traffic on port 25, submit a request to remove the restriction.
   Set smtp.smtp_mail_from to your email address.
   Note: For more information about the preceding configuration options, see [email] and [smtp] on the Apache Airflow website.
5. Create an AWS Secrets Manager secret for the SMTP user and one for the password. Use the Amazon SES SMTP credentials from step 2. Then, configure a startup script to set the environment variables.
   Note: It's a best practice to store your SMTP credentials in Secrets Manager.
6. Add the following startup.sh script to your Amazon Simple Storage Service (Amazon S3) bucket:

   #!/bin/sh
   # Get the SMTP username and password from secrets manager
   username=$(aws secretsmanager get-secret-value --secret-id airflow/variables/smtp.smtp_user --query SecretString --output text)
   password=$(aws secretsmanager get-secret-value --secret-id airflow/variables/smtp.smtp_password --query SecretString --output text)
   
   # Set the SMTP Environment variables with the username and password retrieved from Secrets Manager
   export AIRFLOW__SMTP__SMTP_USER=$username
   export AIRFLOW__SMTP__SMTP_PASSWORD=$password
   
   # Print the SMTP user
   echo "SMTP user is $AIRFLOW__SMTP__SMTP_USER"
   

   Note: Your Amazon MWAA execution role must have permission to retrieve individual secret values.
7. Run the update-environment command to update the Amazon MWAA environment with the email configuration options and the Amazon S3 path for the startup script.

Note: When you create or update the environment, the Apache Airflow callbacks for success, failure, and retry use the preceding configuration. You can also use tasks with EmailOperator to send emails. For more information, see airflow.operators.email on the Apache Airflow website.