I do a crawler to load all my S3 csv files to Glue Data Catalog. Now I want to create a glue job to execute ETL (create and drop temporary tables, select and insert data to tables in Data Catalog) But in the glue job as a python shell I have to split my sql statements to execute one by one. With the following code, I got an error.
client = boto3.client('athena')
client.start_query_execution(QueryString = """drop table if exists database1.temptable1 ;CREATE EXTERNAL TABLE IF NOT EXISTS temptable1(id int ) """, ResultConfiguration = config)
Is there any way to run multiple sql statements in glue job?
AWS公式更新しました 3年前
To be especific, what you want is to run multiple queries on Athena using boto3, doing it from a Glue shell is circumstantial (you could do it from your computer and it would be the same). I don't think it's possible. Maybe try with with pyathena instead