How do I mount an existing Amazon FSx for Lustre file system to AWS Batch in a managed compute environment?

Short description

Use an Amazon Elastic Compute Cloud (Amazon EC2) launch template with AWS Batch. This configuration mounts an existing Amazon FSx for Lustre file system to your containers without the creation of a custom Amazon Machine Image (AMI).

Important: When you create the file system, use the same Amazon Virtual Private Cloud (Amazon VPC) and subnets that are assigned to your compute environment.

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshoot AWS CLI errors. Also, make sure that you're using the most recent AWS CLI version.

To mount an existing Amazon FSx for Lustre file system to AWS Batch in a managed compute environment, complete the following steps:

1. Create an Amazon FSx for Lustre file system. Then, choose either a persistent or scratch file system deployment type.
   Note: Use persistent file systems for longer-term storage and workloads. Use scratch file systems for temporary storage and short-term data processing.

2. Copy your file system ID (for example, fs-0123456789abcdef0). You must have the file system ID to run your launch template.

3. Create a launch template that includes a user data section and uses the MIME multi-part file format. For more information, see Mime multi-part archive on the Cloud-init website.
   See the following Amazon Linux 2 MIME multi-part file example. Replace fs-0123456789abcdef0 with your file system ID, us-east-1 with your AWS Region, and xxxxxxxx with your eight-character Amazon FSx mount name:

   Content-Type: multipart/mixed; boundary="==MYBOUNDARY=="
   MIME-Version: 1.0
   
   --==MYBOUNDARY==
   Content-Type: text/cloud-boothook; charset="us-ascii"
   
   file_system_id=fs-0123456789abcdef0
   region=us-east-1
   fsx_directory=/scratch
   fsx_mount_name=xxxxxxxx
   amazon-linux-extras install -y lustre2.10
   mkdir -p ${fsx_directory}
   mount -t lustre -o noatime,flock ${file_system_id}.fsx.${region}.amazonaws.com@tcp:/${fsx_mount_name} ${fsx_directory}
   
   --==MYBOUNDARY==--

   Note: This example MIME multi-part file configures the compute resource to install the Lustre 2.10 package from the Extras library. The file also mounts an existing Amazon FSx for Lustre file system at /scratch. For installation instructions for other Linux distributions, see Installing the Lustre client.

4. Launch the template to invoke the user data, as shown in the following Amazon Linux 2 example:

   {  "LaunchTemplateName": "user-data",
     "LaunchTemplateData": {
       "UserData": "TUlNRS1W...<base64 encoded userdata>..."
     }
   }
   

   Note: If you add user data to a launch template in the Amazon EC2 console, then make sure that you do one of the following: paste in the user data as plaintext, or upload the user data from a file. If you use the AWS CLI or an AWS SDK, you must first base64-encode the user data. For more information, see Encode to base64 format on the base64encode website. Then, submit that string as the value of the UserData parameter when you call CreateLaunchTemplate.

5. Create a file called mount-fsx-lustre.json.
   Note: Adjust the volume size based on your use case.

6. Run the create-launch-template AWS CLI command to create a launch template based on the mount-fsx-lustre.json file that you created in step 5. Replace us-east-1 with your Region:

   aws ec2 --region us-east-1 create-launch-template --cli-input-json file://mount-fsx-lustre.json

   Example output:

   {  "LaunchTemplate": {
       "LaunchTemplateId": "lt-08cb09d54bcf551f3",
       "LaunchTemplateName": "fsx-test",
       "CreateTime": "2020-06-30T17:13:22.000Z",
       "CreatedBy": "arn:aws:iam::12345678999:user/test",
       "DefaultVersionNumber": 1,
       "LatestVersionNumber": 1
     }
   }

7. Create a new compute environment and associate that environment with your launch template. When AWS Batch spins up instances, the Amazon FSx for Lustre file system is now mounted on the container instances.
   Important: By default, AWS Batch managed compute environments use an approved version of the Amazon Elastic Container Service (Amazon ECS) optimized AMI for compute resources. You must explicitly set the AMI ID for Amazon ECS optimized Amazon Linux 2 AMIs and other Linux distributions.

8. To check whether the file system is mounted with the container instance, use SSH to connect to the instance launched by AWS Batch. Then, run the following Linux df command:

   $ df -h

   Example output:

   Filesystem                                  Size      Used    Avail    Used%   Mounted ondevtmpfs                                    3.9G        0       3.9G     0%       /dev
   tmpfs                                       3.9G        0       3.9G     0%       /dev/shm
   tmpfs                                       3.9G     448K       3.9G     1%       /run
   tmpfs                                       3.9G        0       3.9G     0%       /sys/fs/cgroup
   /dev/xvda1                                   30G     4.2G        25G    15%       /
   172.31.79.79@tcp:/xxxxxxxx                  1.1T     4.5M       1.1T     1%       /scratch
   tmpfs                                       798M       0        798M     0%       /run/user/1000

   Note: /scratch is mounted automatically.

9. Create a job definition in AWS Batch that includes the volume and mount point:

   {  "jobDefinitionName": "Fsx-sample",
     "jobDefinitionArn": "arn:aws:batch:us-east-1:12345678999:job-definition/userdata:1",
     "revision": 1,
     "status": "ACTIVE",
     "type": "container",
     "parameters": {},
     "containerProperties": {
       "image": "busybox",
       "vcpus": 1,
       "memory": 1024,
       "command": [],
       "volumes": [
         {
           "host": {
             "sourcePath": "/scratch"
           },
           "name": "Scratch"
         }
       ],
       "environment": [],
       "mountPoints": [
         {
           "containerPath": "/scratch",
           "sourceVolume": "Scratch"
         }
       ],
       "ulimits": [],
       "resourceRequirements": []
     }
   }

10. To submit an AWS Batch job, use the job definition that you created in step 9.