Athena returns "FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. null"

0

Following the well architected labs 200: Cost and usage analysis I get the following error when adding partitions in Athena Query Editor:

MSCK REPAIR TABLE `cost_optimization_10XXXXXXXX321`;

and it returned the following error:

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. null This query ran against the "costfubar" database, unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: 856e146a-8b13-4175-8cd8-692eef6d3fa5

The table was created correctly in Glue with

Name	cost_optimization_10XXXXXXXXX21
Description	
Database	costfubar
Classification	parquet
Location	
s3://cost-optimization-10XXXXXXX321//
Connection	
Deprecated	No
Last updated	Wed Apr 20 16:46:28 GMT-500 2022
Input format	org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat
Output format	org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat
Serde serialization lib	org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe
Serde parameters	
serialization.format 1
Table properties	
sizeKey 4223322objectCount 4UPDATED_BY_CRAWLER costfubarCrawlerSchemaSerializerVersion 1.0recordCount 335239averageRecordSize 27exclusions ["s3://cost-optimization-107457606321/**.json","s3://cost-optimization-1XXXXXXXX21/**.csv","s3://cost-optimization-107457606321/**.sql","s3://cost-optimization-1XXXXXXXX321/**.gz","s3://cost-optimization-107457606321/**.zip","s3://cost-optimization-107457606321/**/cost_and_usage_data_status/*","s3://cost-optimization-107457606321/**.yml"]CrawlerSchemaDeserializerVersion 1.0compressionType nonetypeOfData file

and has the following partitions shown in Glue:

partition_0
partition_1
year
month
detailed-cur-1XXXXXXXX57
detailed-cur-1XXXXXXXX57
2018
12
View files
View properties
detailed-cur-1XXXXXXXXX57
detailed-cur-1XXXXXXXXX57
2022
4
View files
View properties
detailed-cur-1XXXXXXXXX57
detailed-cur-1XXXXXXXXX57
2018
11
View files
View properties
detailed-cur-1XXXXXXXX57
detailed-cur-1XXXXXXXX57
2018
10
View files
View properties
2 Answers
1

the error was that the prefix in the S3 bucket was empty

answered 2 years ago
0

Can you please check the troubleshooting section here - https://docs.aws.amazon.com/athena/latest/ug/msckrepair-table.html#msck-repair-table-troubleshooting. Its mostly due to permission issues like missing glue:BatchCreatePartition or KMS permissions or s3:GetObject.

AWS
EXPERT
Gokul
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions