How to read the schema only from Glue Data Catalog

0

Hi Team,

I am using below code and its giving me the columns from the data, but my expectation to get the columns from Glue Data Catalog. glueContext.create_dynamic_frame.from_catalog(database = "xyz", table_name = "abc")

Step 1 : Kept 5 columns in a file in S3 and ran the crawler. Glue Data Catalog captured all 5 columns as schema Step 2 : Executed a Glue Pyspark script job with glueContext.create_dynamic_frame.from_catalog and loaded data Iceberg table with merge statement. Step 3 : Removed one column from step1 and Kept 4 columns in a file in S3 as delta file(cdc) and ran the crawler. Glue Data Catalog captured all 4 columns Step 4 : Executed the same Pyspark script job with glueContext.create_dynamic_frame.from_catalog and the job failed with an error : AnalysisException: cannot resolve colunm_name in MERGE command.

Could you please help on this

Thank you in advance !!

Bharath
질문됨 9달 전87회 조회
답변 없음

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠