Firehose error: The schema is invalid. The specified table has no columns.

0

I have a Glue table created from schema in Glue schema registry that is in JSON draft 07 format. I tried setting up a Firehose that converts incoming JSON data into Parquet as it saves it into S3, given the Glue table.

If I create a Glue table by giving custom column definitions that are exactly the same as in the table that is created from Glue schema - it seems to work fine, with the exception that I need to run a crawler to make data available in Athena.

I also found statement on SO "...KDF is unable to read table's schema if table is created from existing schema." - is this true? could someone verify this or is there another way to make this work with schema supplied as JSON draft 07?

hRed
asked 6 months ago190 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions