Migrate on-prem data( files) to S3 using Snowball and data sync using DataSync

0

Customer wants to migrate 900 TB of unstructured data( file data) and this data continues to change/update. We are thinking to use snowball for the data migration and datasync to sync the incremental data after the migration. Can we use DataSync to transfer any incremental changes after the migration (something like DMS used to migrate database CDC) ? what are the other options to sync the incremental changes ?

1개 답변
1

There are a few different options in this case and I definitely recommend reaching out to your AWS account contacts so they can offer more assistance.

Firstly the use of Snowball will depend on the speed of your network links, the rate of data change, migration cutover window and data security requirements. eg 900TB can be transferred over a multi tunnel vpn using s3 multipart upload if your internet links are good, security policies allow and the rate of change isn't high. The aws s3 sync command can subsequently perform sync tasks. If you have to use Snowball standard s3 commands can be used to copy the device to snowball and subsequently sync (depending on number this can take a lot of time) but you do not have to use DataSync

DataSync as you suggested is something to consider and can also be used with Snow appliances. If you need to run the sync tasks on a defined schedule, control the bandwidth and task duration it should definitely be considered over standard s3 cli commands as offers more options and optimisations....and yes it handles incremental (only transfers changed files). See this blog

The other main option to consider is the Storage Gateway appliances. If you are going to be operating in a hybrid mode with data both onpremise and in AWS for an extended period of time then this becomes an attractive option

AWS
전문가
Peter_G
답변함 2년 전
profile picture
전문가
검토됨 24일 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인