Unable to Create Table with AWS Wrangler: NoSuchKey Error with Athena Iceberg

0

I'm encountering a NoSuchKey error while attempting to create an Athena Iceberg table using AWS Wrangler. Here's the anonymized error message:

{ "errorMessage": "GENERIC_INTERNAL_ERROR: io.trino.hdfs.s3.TrinoS3FileSystem$UnrecoverableS3OperationException: com.amazonaws.services.s3.model.AmazonS3Exception: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: VNND5RES7B2N8ZQ3; S3 Extended Request ID: V26qT2lZsbEWk1R2Met9VWO8XiKIM3TDDORqkV7394OHBHvhD1WV85cjAgExRPkjR7DWKwmvwv4=; Proxy: null), S3 Extended Request ID: V26qT2lZsbEWk1R2Met9VWO8XiKIM3TDDORqkV7394OHBHvhD1WV85cjAgExRPkjR7DWKwmvwv4= (Bucket: [REDACTED], Key: [REDACTED]). If a data manifest file was generated at '[REDACTED]', you may need to manually clean the data from locations specified in the manifest. Athena will not delete data in your account.", "errorType": "QueryFailed", "requestId": "35e8555b-40dd-4a69-8b0e-1e76ed02828b", "stackTrace": [ " File "/var/task/main.py", line 325, in handler\n process_data(file=file, account_id=account_id)\n", " File "/var/task/main.py", line 301, in process_data\n merge_insert(\n", " File "/var/task/main.py", line 215, in merge_insert\n wr.athena.to_iceberg(\n", " File "/var/lang/lib/python3.11/site-packages/awswrangler/_config.py", line 735, in wrapper\n return function(**args)\n", " File "/var/lang/lib/python3.11/site-packages/awswrangler/_utils.py", line 178, in inner\n return func(*args, **kwargs)\n", " File "/var/lang/lib/python3.11/site-packages/awswrangler/athena/_write_iceberg.py", line 411, in to_iceberg\n wait_query(query_execution_id=query_execution_id, boto3_session=boto3_session)\n", " File "/var/lang/lib/python3.11/site-packages/awswrangler/_config.py", line 735, in wrapper\n return function(**args)\n", " File "/var/lang/lib/python3.11/site-packages/awswrangler/athena/_executions.py", line 236, in wait_query\n raise exceptions.QueryFailed(response["Status"].get("StateChangeReason"))\n" ] } How can I address this issue and successfully create a table using AWS Wrangler with the Athena Iceberg format?

This is happening for all tables in all databases.

I've noticed that the file specified in the error message is never created, and it's supposed to be generated by Iceberg, not manually. Additionally, when attempting to run OPTIMIZE cobo REWRITE DATA USING BIN_PACK and msck repair table, I receive a "Query type not supported by Athena Iceberg at this time" error.

Even running the following doesn't work:

CREATE TABLE iceberg_table ( id int, data string, category string) PARTITIONED BY (category, bucket(16,id)) LOCATION 's3://athena-iceberg-prod/iceberg_table/' TBLPROPERTIES ( 'table_type'='ICEBERG', 'format'='parquet', 'write_compression'='snappy', 'optimize_rewrite_delete_file_threshold'='10' )

The above works, creating the table. But the following (after a long time) fails

INSERT INTO iceberg_table VALUES (1,'a','c1')

GENERIC_INTERNAL_ERROR: io.trino.hdfs.s3.TrinoS3FileSystem$UnrecoverableS3OperationException: com.amazonaws.services.s3.model.AmazonS3Exception: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: ND190V3KAYN5P84S; S3 Extended Request ID: TsstDwqfvG4H6mJ1/+G3y4eCGaw+a2mEgtgt0s9A/1Y5cqU/lwB61Cpk=; Proxy: null), S3 Extended Request ID: TsstDwqfvG4H6mJ1/+G3y4eCGaw+a2mEgtgt0s9A/1Y5cqU/lwB61Cpk= (Bucket: athena-iceberg-prod, Key: iceberg_table/metadata/00000-3b22f196-9f51-455c-8cfd-f7431897fd39.metadata.json).

Could this be a permission error?