How to estimate size and number of DMS instances for a given number of ongoing replications
A customer is looking to use DMS to replicate 20 SQL Server 2016 Databases, ongoing to S3 for their Datalake. I am looking for advice/guidance on
- what replication instance size I should recommend
- How many replication instances they should use to replicate 20 databases ongoing The customer estimates that each databases only has a few thousand transactions per day (they're not hugely high throughput databases). If anyone has any guidance beyond the very general guidance offered in the DMS User Guide, I would be very grateful!
- 최신
- 최다 투표
- 가장 많은 댓글
General guidance starts with suggesting customer not to use t2 instance type. Because t2 instance types are designed for light workloads with occasional bursts in performance. We recommend using this instance class to learn about DMS and do test migrations of small, intermittent workloads.
There is no such formula based on which someone can identify the size of replication instance. Because it is entirely dependent on few points like below:
-Amount of tables to be loaded in parallel (during full load). By default DMS migrates 8 tables in parallel per task, you can either increase or decrease the number of tables to be migrated as per the size of replication instance. Or alternatively you can set the maximum number of tables to be loaded in parallel based on the resources (memory, CPU) on replication instance.
-Important point to note here is, even if customer starts with a lower size replication instance. And later plans to upgrade the replication instance to a higher instance type, it can be done seamlessly. Replication tasks running on the replication instance will be stopped and resumed automatically (without manual intervention) once the scale compute is complete.
-Once the full load is complete, then resources will be utilized by the amount of data being pushed on source database. DMS reads data from source database by running queries (in case of sql server) on transaction logs and push the filtered changes to target database.
-DMS will only store those changes on replication instance in case if DMS is unable to push changes to target in the same speed as they are being applied on source database.
It is always recommended to monitor the replication instance resources and scale them as per the resource utilization. There are some guidelines in choosing the right replication instance type in https://aws.amazon.com/blogs/database/aws-dms-now-supports-r4-type-instances-and-learn-to-choose-the-right-instance-class-for-migrations-using-aws-dms/ which should be your starting point.
Lastly, there is a blog about monitor DMS instance metrics and scale it dynamically. https://aws.amazon.com/cn/blogs/database/how-to-scale-aws-database-migration-service-dms-replication-instances/
Please let me know if you have any questions on this.
