Athena returns "FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. null"

0

Following the well architected labs 200: Cost and usage analysis I get the following error when adding partitions in Athena Query Editor:

MSCK REPAIR TABLE `cost_optimization_10XXXXXXXX321`;

and it returned the following error:

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. null This query ran against the "costfubar" database, unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: 856e146a-8b13-4175-8cd8-692eef6d3fa5

The table was created correctly in Glue with

Name	cost_optimization_10XXXXXXXXX21
Description	
Database	costfubar
Classification	parquet
Location	
s3://cost-optimization-10XXXXXXX321//
Connection	
Deprecated	No
Last updated	Wed Apr 20 16:46:28 GMT-500 2022
Input format	org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat
Output format	org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat
Serde serialization lib	org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe
Serde parameters	
serialization.format 1
Table properties	
sizeKey 4223322objectCount 4UPDATED_BY_CRAWLER costfubarCrawlerSchemaSerializerVersion 1.0recordCount 335239averageRecordSize 27exclusions ["s3://cost-optimization-107457606321/**.json","s3://cost-optimization-1XXXXXXXX21/**.csv","s3://cost-optimization-107457606321/**.sql","s3://cost-optimization-1XXXXXXXX321/**.gz","s3://cost-optimization-107457606321/**.zip","s3://cost-optimization-107457606321/**/cost_and_usage_data_status/*","s3://cost-optimization-107457606321/**.yml"]CrawlerSchemaDeserializerVersion 1.0compressionType nonetypeOfData file

and has the following partitions shown in Glue:

partition_0
partition_1
year
month
detailed-cur-1XXXXXXXX57
detailed-cur-1XXXXXXXX57
2018
12
View files
View properties
detailed-cur-1XXXXXXXXX57
detailed-cur-1XXXXXXXXX57
2022
4
View files
View properties
detailed-cur-1XXXXXXXXX57
detailed-cur-1XXXXXXXXX57
2018
11
View files
View properties
detailed-cur-1XXXXXXXX57
detailed-cur-1XXXXXXXX57
2018
10
View files
View properties
2 Antworten
1

the error was that the prefix in the S3 bucket was empty

beantwortet vor 2 Jahren
0

Can you please check the troubleshooting section here - https://docs.aws.amazon.com/athena/latest/ug/msckrepair-table.html#msck-repair-table-troubleshooting. Its mostly due to permission issues like missing glue:BatchCreatePartition or KMS permissions or s3:GetObject.

AWS
EXPERTE
Gokul
beantwortet vor 2 Jahren

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen