Lake Formation Governed tables: truncate and reload?

Question

Doing a POC with LF Governed table for a small dimension. I'd like to purge and reload the table each time it runs. I have tried to include a purge_table command within a governed table transaction but I receive the below error. I've tried adding a transactionId param to the purge_table command directly or within the options dict and neither has worked. Any ideas? Or anyone found a good way to truncate and reload a governed table?

"An error occurred while calling o119.purgeTable. Specify Either Transaction Id or Query AsOf Time"

Answer

Hello,

I tried to purge the table and reload but unable to reload the table. Could you please share the purge_table command you are using for us to test ? Also, we recommend you to please open a support ticket with us so that we can get on a screen-share and assist you by checking your Lake Formation configuration and parameters used.

Answer

In my POC to work with a clean LF table, I do the following:
```
import awswrangler as wr

# delete table if exists
wr.catalog.delete_table_if_exists(database=DATABASE, table=TABLE)

# clean up S3 location (delete_table do not remove data itself, just a metadata)
wr.s3.delete_objects(TABLE_LOCATION)

#Create a test table
wr.catalog.create_parquet_table(
        database=DATABASE,
        path=TABLE_LOCATION,
        table=TABLE,
        partitions_types = partition_types,
        columns_types=columns_types,
        description='Table for testing lakeformation',
        table_type = "GOVERNED",
        compression= 'snappy'
    )
```

Lake Formation Governed tables: truncate and reload?

Relevanter Inhalt