- Newest
- Most votes
- Most comments
Hi Santosh,
Thank you very much.
Where can I find out which providers version my airflow is using? I want to make sure I am looking at the right documents and continue to look at the right documents.
How can I tell that I should use apache-airflow-providers-amazon/6.0.0 and not the current stable version, https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/_api/airflow/providers/amazon/aws/operators/s3/index.html?
Hi,
The module named 'airflow.providers.amazon.aws.operators.s3_bucket' was used until apache-airflow-providers-amazon version 2.4.0. Version 2.4.0 used to be the default amazon provider package version installed in MWAA with Airflow v2.0.2. Based on the error that you are seeing, I am suspecting that you are working on the MWAA environment with the latest version (2.4.3) where the he default amazon provider package version installed is 6.0.0.
s3_bucket module in v2.4.0 https://airflow.apache.org/docs/apache-airflow-providers-amazon/2.4.0/_api/airflow/providers/amazon/aws/operators/s3_bucket/index.html
Same module is not present in v6.0.0 and in the immediate releases after 2.4.0, we can see the message that it is deprecated https://airflow.apache.org/docs/apache-airflow-providers-amazon/3.0.0/_api/airflow/providers/amazon/aws/operators/s3_bucket/index.html "This module is deprecated. Please use airflow.providers.amazon.aws.operators.s3."
Please use the operators of 6.0.0 as mentioned in the following document using the "airflow.providers.amazon.aws.operators.s3" instead. https://airflow.apache.org/docs/apache-airflow-providers-amazon/6.0.0/_api/airflow/providers/amazon/aws/operators/s3/index.html
If you are not using MWAA environment with the latest version 2.4.3, I would request you to check the specific operator documentations to see the correct modules as per the apache-airflow-providers-amazon version.
Thank you.
Relevant content
- asked 5 months ago
- asked a year ago
- Accepted Answerasked 5 months ago
- asked 5 years ago
- AWS OFFICIALUpdated 2 months ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated a year ago
MWAA environments with Airflow v2.2.2 (you can check the Airflow version in the console) comes with apache-airflow-providers-amazon package version 2.4.0. With Airflow v2.4.3, the default version installed in 6.0.0.
You can check the installed versions by running the "pip3 list" command using BashOperator task similar to the example DAG below
I hope the above information helps. Please feel free to write back if you any other queries or concerns, I will be more than happy to help.
Thank you.
I hope the above information helps. Please feel free to write back if you any other queries or concerns, I will be more than happy to help.
Thank you.