How do I create a fixed-width classifier manually for a Glue crawler?
0
I have a few text files on S3 that I need to add to the Glue Catalog in order to use them in a job. None of them have separators, they are all fixed-width files. I have the schemas, but the crawler naturally fails to recognize them. I tried processing the files using a Lambda function, but since they are too big (one is 11GB), execution times out, even at the maximum available (15min).
I couldn't find any tutorials about dealing with these type of files.
asked 3 months ago99 views
1 Answer
- Newest
- Most votes
- Most comments
1
Hi,
Go to this article to find a fully explained example: https://medium.com/@avj70380/how-to-use-grok-custom-classifier-to-read-flat-text-files-using-aws-glue-657c7d55bd57
Best,
Didier
Relevant content
- asked 2 years ago
- asked 2 years ago
- Accepted Answerasked 3 years ago
AWS OFFICIALUpdated 3 years ago- AWS OFFICIALUpdated 3 years ago
- How can I use a Lambda function to automatically start an AWS Glue job when a crawler run completes?AWS OFFICIALUpdated 2 years ago