By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Load S3 Bucket data into Aurora PostgreSQL tables

0

I need to load my data from S3 Bucket into the Aurora PostgreSQL tables. I read https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_PostgreSQL.S3Import.html#USER_PostgreSQL.S3Import.Reference this documentation and I will try. But, do we have any other way to be able to handle this case?

Especially I want to ask, Is there any LOAD command which exists on Aurora PostgreSQL?

Thanks!

Topics
Migration & ModernizationStorageDatabaseAnalytics
Tags
Migration & ModernizationAmazon Simple Storage ServicePostgreSQLAurora PostgreSQLAmazon Aurora
Language
English
keyserSozee
asked 2 years ago862 views
2 Answers
1

One option could be to use DMS.

DMS does support S3 as a source - https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.S3.html

DMS supports Aurora Postgres as a target - https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.PostgreSQL.html

profile pictureAWS
EXPERT
Indranil Banerjee AWS
answered 2 years ago
  • Mayura
    a year ago

    I want to perform integrity check on my files in S3 bucket (The record count and md5 are 2 checks we need to do when files arrive in S3 from source DB). When these checks are successful, we want to load these files into Postgres. Can you please guide how this can be done? Integrity checks can be done in a bash script. Can we use the same script to copy files to Postgres?

0

The procedure for using the table_import_from_s3() function is explained here: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_PostgreSQL.S3Import.html

You call the function from a SELECT statement. And so you could automate that by calling psql from a shell script, and passing the SELECT statement as a -c (command) or -f (SQL script file) argument to psql.

AWS
AWS-User-0081273
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content