Trigger Glue Crawler with S3 bucket event

0

Hi , I would like to trigger Glue Crawler , once the S3 bucket specific prefix files are being added. Is there any design pattern I can follow, seeking guidance from the experts.

I have gone through this blogpost : https://aws.amazon.com/blogs/big-data/run-aws-glue-crawlers-using-amazon-s3-event-notifications/

But I am looking for more simpler solution than extra resource creation. Thanks,

1回答
0
承認された回答

Hello.

Since the S3 event trigger cannot directly trigger the Glue crawler, you need to create a configuration using SNS or SQS as per the documentation you checked.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/EventNotifications.html

I think it is also possible to run the Glue crawler using Glue workflow and EventBridge event using methods other than S3 event triggers.
When using EventBridge event, there is no need to create SNS or SQS.
https://docs.aws.amazon.com/glue/latest/dg/workflows_overview.html

profile picture
エキスパート
回答済み 7ヶ月前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ