Crawler does not allow to change from recrawl all to only new partitions in aws Glue

0

When you have multiple datasources in a crawler it does not allow to change the crawling from all paritions again to just new partition

KG
preguntada hace 6 meses218 visualizaciones
1 Respuesta
0

Hi KG,

It is in fact possible for AWS Glue to crawl only new partitions that are added in Amazon S3 folders since the last crawl. If the schemas are compatible, new partitions will be added to existing tables. To do so, click on the name of your crawler > edit. Under Step 2: Choose data sources and classifiers, click edit. Select the Data Source of interest and click edit. From there, you should be able to select the option Crawl new sub-folders only. Hope this helps!

AWS
respondido hace 6 meses
  • Hi I did the same steps you mentioned , that is when i get the error where it does not update the crawlering to new partitions only

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas