Is it possible to ingest large-scale data from a Snowflake Enterprise-tier account (to RDS) using an AWS Glue Spark job and without using a NAT Gateway?

Question

I want to ingest a large table from my organisation's Snowflake database into an Aurora V2 Serverless Postgresql database. I am looking at using an AWS Glue Spark job for this process. However, the Snowflake account tier we are on is not PrivateLink-enabled. We also do not want to use a NAT Gateway as the costs in the long term will be too high. Is there any way of reading directly from Snowflake using AWS Glue, without using either VPC endpoints or a NAT Gateway to route the traffic?

Answer

If not using privatelink, you should be able to access it adding an internet gateway to your Glue VPC. If that's not allowed in the VPC you need for postgress, then you might need to do something more complicated like peering them.

Is it possible to ingest large-scale data from a Snowflake Enterprise-tier account (to RDS) using an AWS Glue Spark job and without using a NAT Gateway?

Contenus pertinents