Glue Crawler and Classifiers - Supported file encodings: is UTF16 supported?

Question

Hi,
AWS Glue Crawlers with CSV and XML Classifiers and  works well with files encoded in UTF-8 but not with file encoded in UTF-16.

Public documentation does not clarify this point:
 
 - Do Glue crawler and classifier support UTF-16?
 - Is there please an available documentation on supported encodings with Glue crawlers and classifiers?

Best regards,

Nicolas.

Accepted Answer

Glue at the moment supports UTF-8 encoded files only [1]. If UTF-16 files are passed in, you may encounter the "Internal Service Exception" error message. The most feasible method would be to programatically convert the utf-16 files to utf-8 before passing it through Glue Crawler.

[1] - https://docs.aws.amazon.com/glue/latest/dg/components-key-concepts.html

Glue Crawler and Classifiers - Supported file encodings: is UTF16 supported?

相关内容