When trying to insert data into an Athena/Iceberg table, using Java Athena JDBC Driver 3.0.0, getting error "Member must have length less than or equal to 1024".
The table:
CREATE TABLE IF NOT EXISTS events (id STRING, event_timestamp_utc TIMESTAMP, event_type STRING, source STRING, source_timestamp TIMESTAMP, rawdata STRING,media STRING)
PARTITIONED BY (month(source_timestamp), bucket(10, id))
LOCATION 's3://" + bucket + "/athena-iceberg/'
TBLPROPERTIES ( 'table_type' ='ICEBERG', 'optimize_rewrite_data_file_threshold' = '49' )
The insert:
INSERT INTO events (id, event_timestamp_utc, event_type, source, source_timestamp, rawdata, media) VALUES (?, ?, ?, ?, ?, ?, ?)
Why does this limitation exist? Even SQLITE doesn't have such small arbitrary limits.
Is there a workaround?