1개 답변
- 최신
- 최다 투표
- 가장 많은 댓글
0
I've tested a crawler using the same folder structure in S3 as mentioned.
Specified include path as: s3://my-datalake/projects/
Exclude pattern as: incremental_**/**
Using above exclude pattern ignores all files under folders named 'incremental_'. The only additional thing could be that existing crawlers have "UpdateBehavior" as "LOG" - so the already created tables are not being dropped. You could try updating it to "UPDATE_IN_DATABASE" - this will recreate the tables.
Reference - https://docs.aws.amazon.com/glue/latest/dg/define-crawler.html#crawler-data-stores-exclude
답변함 일 년 전