Copying files from SMB shares into S3 using either FSx for Windows File Server or S3


So trying to see which would be a good solution to implement and also a good user experience for the end user.

I want to move off some old unused data away from my Windows file servers, accessible via SMB, into S3 then to S3 Glacier. Was thinking of using FSx for Windows File Server then use a lifecycle policy to off load it into Glacier, reason using this FSx was to preserve the permissions of the files and folders.

Would I be able to achieve the same outcome but using S3 instead as FSx for Windows File Server is expensive as heck!

profile picture
asked a year ago1633 views
2 Answers

Amazon FSx for Windows File Server is designed to provide fully managed, highly reliable, and scalable file storage that is accessible over the industry-standard Server Message Block (SMB) protocol. It's fully compatible with Windows and provides features like user quotas, end-user file restore, and Microsoft Active Directory integration. However, it is indeed more expensive compared to other services like Amazon S3.

Amazon S3 is a simple storage service that offers industry-leading scalability, data availability, security, and performance. S3 can be integrated with AWS Glacier to archive data in a cost-effective manner. However, it's important to note that S3 does not support SMB directly, and you would need to use something like an Amazon EC2 instance to facilitate this connection.

S3 does not natively support the same permissions model as an SMB file share. S3 uses a bucket and object-based permissions model, rather than the NTFS file permissions model used by SMB. However, you can use AWS Identity and Access Management (IAM) to manage access to your S3 resources.

If maintaining exact NTFS permissions is important, FSx may be your best option despite its higher cost. If you're willing to adapt to the S3 permissions model and manage file access through IAM, you could save costs by using S3.

To transfer files directly from your SMB shares to S3, you can use AWS DataSync. AWS DataSync is a data transfer service that makes it easy for you to automate moving data between on-premises storage and Amazon S3 or Amazon Elastic File System (Amazon EFS). You can use DataSync to migrate active datasets, archive data, replicate data for business continuity, or transfer data to the cloud for analysis and processing.

To automate moving data from S3 to Glacier, you can set up a lifecycle policy on your S3 bucket. A lifecycle policy is a set of rules that define actions that Amazon S3 applies to a group of objects. These rules can make transitions between different storage classes, for example from STANDARD to GLACIER to optimize costs.

So while FSx for Windows File Server may be a more direct solution in terms of preserving SMB shares with their existing permissions, a combination of S3, IAM, DataSync, and S3 Lifecycle Policies can achieve similar results at a potentially lower cost. But it will require more initial setup and possibly adaptation by your end users.

You could read the following CloudFromation template from the AWS Samples repo to understand the configuration required to move data from SMB to S3: AWS DataSync CloudFormation Sample - SMB to S3

profile picture
answered a year ago

Based on your brief description, I'd explore using an S3 file gateway which you can spin up on EC2, and not necessarily have on-prem. You can use that for your SMB access to copy data, spin it down when not in use. Your data would then be on S3 and you could use lifecycle policies to move that over to glacier. Since this may be a one time move to get off the old server, this might work for you.

It doesn't sound like you require all of the fully functioned Windows server environment - so I'd shy away from the FSx for Windows.

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions