crawler cannot get snowflake table

Question

Hi, i have a crawler that connects by jdbc to a snowflake and the process finishes well, but it doesn't create any table, here the output of the cloudwatch:

BENCHMARK : Classification complete, writing results to database correosmaskdb
 INFO : Crawler configured with SchemaChangePolicy {
    "UpdateBehavior": "UPDATE_IN_DATABASE",
    "DeleteBehavior": "DEPRECATE_IN_DATABASE" }
 BENCHMARK : Finished writing to Catalog
 BENCHMARK : Crawler has finished running and is in state READY

the connector is working fine, if i do a test connection it shows that it was successfull
"Successfully connected to the data store with connection snowflake-glue-jdbc-connection2."

so, i don't know where is the issue and why the logs shows that it has written to catalog when in fact it has not, thanks for your help

Accepted Answer

In the crawler output record (not the log) do you see the number of tables that were created?

From the limited info from the logs it seems that the Crawler completed correctly.

So either:

1. the crawler has created 0 tables , this should mean that the inclusion pattern may be incorrect and pointing to a Not existing SnowFlake Schema , hence no table found and no table created in the catalog.
2. the crawler has created > 0 tables, that this should mean that the role used by the crawler has the right privileges on the target datatabase, but the role you are using to look at the catalog does not have the right privileges to list the tables.

can you provide some additional information on the number of tables created by the crawlers (not looking at the Db but at the crawler output page), and the privileges the role used to access the glue console has?

thank you

Answer

Based on the logs you provided, it seems like your crawler has completed successfully and is in the "READY" state, but it's not clear if any tables were actually created.

One possibility is that the crawler is configured to only update existing tables and not create new ones. You can check the "SchemaChangePolicy" configuration in the logs to see if this is the case.

Another possibility is that there was an error or issue when creating the tables, but this was not captured in the logs. You can try running the crawler again and checking the logs for any errors or exceptions that may have occurred.

It's also possible that the tables were created in a different schema or location than you were expecting. You can check the Snowflake database to see if any new tables were created and their location.

crawler cannot get snowflake table

相关内容