Glue crawled tables showing in glue's table (with the right schema) but don't show in athena, explicitly query results in hive format error

0

I crawled a postgres database using JDBC crawler and it successfully created tables and is showing them in glue > database > "db_name" > Tables. It have fetched in columns all right.

Not athena show 0/zero tables for this database and if I write a query using 'db_name'.'table_name', it gives following error.

HIVE_UNSUPPORTED_FORMAT: Unable to create input format This query ran against the "facset-loader" database, unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: fc373521-3501-442d-b5a2-eb583007bcc1.

But how come the created tables have format errors when they show perfectly in the glue.

質問済み 2年前663ビュー
1回答
0

I understand that you have a table created by crawler but when you tried to query the table from Athena, you were getting the HIVE_UNSUPPORTED_FORMAT: Unable to create input format. The steps that you have mentioned would be applicable only if the table that you want to query is present in s3 but your data source is postgres database. This must be the reason why you were unable to view the table in Athena. As you have mentioned that you are trying to query a data source which is a postgres database, you will have to follow a different type of mechanism for the sake of running queries against your postgres database in Athena. This mechanism known as the Athena Federated Querying. Please have a look at this documentation to know more about Federated queries in Athena.

I have found a third party blog which has detailed steps that you need to follow in order to query postgre database from Athena. In the blog, it is highlighted that in order to view the table in Athena console, you will have to add a new environment variable to the Lambda function.

Please try following this and see whether it works for you or not.

I hope the information provided by me would be of use to you.

profile pictureAWS
サポートエンジニア
Chaitu
回答済み 2年前
AWS
エキスパート
レビュー済み 2年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ