Why redshift table structure getting changed while loading with glue job?

0

Redshift table structure(schema) got changed after loading data with glue. Why this is happening..

  • You mean the column types changed or even the number of columns/order?

  • Columns data type changed and even number of columns increased.

asked a year ago213 views
1 Answer
0

The reason for that is that for efficiency the table is dumped to the s3 temporary path specified as CSV, this means some specific types are lost and it's possible that some columns have characters that make it incorrectly parse as more columns that it really has.

Glue 4.0 has a new improved connector to Redshift, can you still reproduce it in that version? If that's the case, please open a support case or indicate here how to reproduce the issue with the multiple column (e.g. what character is confusing the CSV parsing).

profile pictureAWS
EXPERT
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions