Memory requirements

0

Hi
Hopefully a quick question, but are the memory system requirements flexible?

https://docs.aws.amazon.com/datasync/latest/userguide/requirements.html#hardware

I can get a VM with 16GB of ram, but 32GB isn't possible at this time.

Will the data sync agent run with 16GB?
Is there setting that might need to be tweaked?

I'm not that concerned with the highest performance, we will likely need to cap the transfer at 100Mbps

Cheers,
Jason

질문됨 5년 전330회 조회
5개 답변
0

Hi Jason,
Thanks for reaching out. Using 16GB of RAM will work for transferring less than 10 million files per task. However, you may run into other limits or lower performance. I've also noted this as a feature request for lower memory requirements. Please let us know if you have additional feedback, and feel free to send me a private message if you have more questions about your specific account/setup.
Regards,
Olga Kogan
AWS DataSync

답변함 5년 전
0

Hi.
Thanks for the response.
We have about 25 million files total, about 9TB.
Each task would copy over around 6 million files, 1.8TB.

Each task would copy less than 10 million, after the second task there would be more than 10 million files in the S3 bucket.
With this scenario, do you think 16GB would still be workable?

We are doing a full systems migration to AWS, as such we can not purchase any new hardware as much of our existing infrastructure will be freed up once the migration is done.
But without the data being migrated, the servers can't be made available.
Bit of a chicken and the egg problem!

One other question, for the logging, is there any way to capture the list of files that were copied?

Cheers,
Jason

답변함 5년 전
0

Hi Jason,
As I mentioned before, using 16GB of RAM will work for transferring the number of files per task you described. However, as this is not our recommended configuration, you may run into other limits or lower performance. If you run into issues, please contact support and we'll take a look. We don't currently generate a list of all transferred files. I've noted your suggestion and we'll consider it as we plan new features.
Regards,
Olga Kogan
AWS DataSync

답변함 5년 전
0

Hi Olga,
Thanks again for your reply.

We are going to give it a try and see how it goes.

I have put in a request for more memory, maybe I'll get lucky!

Thanks for noting my requests for features, both would be welcome additions.

I know it is early in the product life cycle.

Cheers,
Jason

답변함 5년 전
0

Hi Jason,

We have just released the ability to log all transferred files:
https://aws.amazon.com/about-aws/whats-new/2020/04/aws-datasync-enhances-monitoring-capabilities-with-file-level-logging/

Best,
Olga

답변함 4년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠