How can I transfer my data to my AWS Snowball Edge device?

6 minute read
0

I'm using AWS Snowball to transfer my data to AWS. I want to transfer my data onto my Snowball Edge device.

Short description

You can transfer data onto a Snowball Edge device using one of the following methods:

As an example, follow these steps to set up your Snowball Edge device for transferring data using the Amazon S3 Adapter for Snowball:

  1. Get the device's unlock code and manifest file.
  2. Configure and unlock the Snowball Edge device.
  3. Configure a profile on the AWS Command Line Interface (AWS CLI) to use with the Snowball Edge device.
  4. Transfer data using the Amazon S3 Adapter for Snowball.

Resolution

Get the device's unlock code and manifest file

To get the device's credentials using the Snowball console, see Get your credentials and tools.

Note: You can use the Snowball console to get the device's credentials only after the device is delivered to you.

To get the device's credentials using the AWS CLI, do the following:

1.    Install AWS CLI version 1.16.14 or earlier. Only these AWS CLI versions currently support use with Snowball Edge devices.
Note: You can choose to uninstall this CLI version and install AWS CLI version 2 or later after the data transfer is complete.

2.    Run the list-jobs command to get the Snowball job ID:
Note: Replace the value for --region with the AWS Region that your Snowball job is in.

aws snowball list-jobs --region us-east-1

3.    The command returns output similar to the following:

{
      "JobId": "JIDexample-aaaa-1111-bbbb-222222222222",
      "JobState": "WithCustomer",
      "IsMaster": false,
      "JobType": "IMPORT",
      "SnowballType": "EDGE",
      "CreationDate": 1532615527.2,
      "Description": "My_First_Snowball_Job"
 }

Copy the JobID. If you have multiple jobs, then all the jobs are listed. Be sure to copy the JobID for the correct job.

4.    Run the get-job-unlock-code command to get the unlock code:

aws snowball get-job-unlock-code --job-id  JIDexample-aaaa-1111-bbbb-222222222222

5.    The command returns the unlock code, which is 25 alphanumeric characters separated by four dashes, similar to the following:

01234-abcde-01234-ABCDE-01234

6.    Run the get-job-manifest command to get the manifest file:

aws snowball get-job-manifest --job-id JIDexample-aaaa-1111-bbbb-222222222222

7.    The command returns an Amazon S3 presigned URL for the job's manifest file. Store the location path where you download the manifest file—you need this path for a later step.

Configure and unlock the Snowball Edge device

1.    Download and install the Snowball client.

2.    Using the Snowball client, run the configure command from your workstation to store your job's credentials (manifest and unlock code) as a profile:

snowballEdge configure

3.    The command returns prompts to enter information about the device and job. Replace the following values in the example prompts:
Snowball Edge Manifest Path: Enter the path to the manifest file.
Unlock Code: Enter the unlock code.
Default Endpoint: Enter the IP address of the Snowball Edge device.

Configuration will stored at home directory\.aws\snowball\config\snowball-edge.config
Snowball Edge Manifest Path: /Downloads/JID2EXAMPLE-0c40-49a7-9f53-916aEXAMPLE81-manifest.bin
Unlock Code: 01234-abcde-01234-ABCDE-01234
Default Endpoint: https://192.0.2.0

Warning: Users who can access the configuration file can also access the data on your Snowball Edge devices or clusters. You must manage local access control for this file.

4.    Run the unlock-device command to unlock the Snowball Edge device:

snowballEdge unlock-device

5.    After you run the unlock-device command, you get a message that includes "Your Snowball Edge device is unlocking." To refresh the status, run the describe-device command:

snowballEdge describe-device

6.    After the state of your device becomes "UNLOCKED", you can copy data to the device.
Note: If you get errors when you try to unlock the device, see Troubleshooting connection problems.

Configure a profile on the AWS CLI to use with the Snowball Edge device

Note: Only AWS CLI versions 1.16.14 and earlier currently support Snowball Edge devices.

1.    Use the Snowball client to get your local credentials for use with the AWS CLI. Run the list-access-keys command to get the access key ID:

snowballEdge list-access-keys

2.    The command returns the value for AccessKeyIds. Run the get-secret-access-key command against your access key ID:

snowballEdge get-secret-access-key --access-key-id EXAMPLEACCESSKEYID

3.    The command returns both the access key and the secret access key, similar to the following:

[snowballEdge]
aws_access_key_id = EXAMPLEACCESSKEYID
aws_secret_access_key = EXAMPLEKEYaBc/D1EFGHI/jKlMnoPQRSTUVXYZAB

4.    Open the AWS CLI. Then, run the configure command with the --profile argument to configure a profile on the AWS CLI:

aws configure --profile snowballEdge

5.    The command returns prompts to enter the credentials for the Snowball Edge profile. Replace the following values in the example prompts:
AWS Access Key ID [None]: Enter the access key ID returned by the Snowball client.
AWS Secret Access Key [None]: Enter the secret access key returned by the Snowball client.
Default region name [None]: snow
Default output format [None]: json

AWS Access Key ID [None]: EXAMPLEACCESSKEYID
AWS Secret Access Key [None]: EXAMPLEKEYaBc/D1EFGHI/jKlMnoPQRSTUVXYZAB
Default region name [None]: snow
Default output format [None]:

For more information, see Using the AWS CLI and API operations on Snowball Edge.

Transfer data using the Amazon S3 Adapter for Snowball

Important: For the value of --endpoint in the following commands, enter the IP address of the Snowball Edge device appended with :8080. For the value of --profile, use the named profile that you created on the AWS CLI for use with the Snowball Edge device.

To transfer a single file to the device, run the s3 cp using the AWS CLI:

aws s3 cp source/path/example/text.txt s3://destination-bucket/path/example --endpoint http://192.0.2.0:8080 --profile snowballEdge

To transfer directories or multiple files, run the s3 cp command with the --recursive option:

aws s3 cp --recursive source/path/example/ s3://destination-bucket/path/example --endpoint http://192.0.2.0:8080 --profile snowballEdge

To remove objects from the device, run the s3 rm command:

aws s3 rm s3://destination-bucket/path/example --endpoint http://192.0.2.0:8080 --profile snowballEdge

To list the files already transferred to the device, run the s3 ls command:

aws s3 ls s3://destination-bucket/path/example --endpoint http://192.0.2.0:8080 --profile snowballEdge

It's a best practice to validate the data during and after the transfer. For more information, see Data validation with Snowball Edge jobs.


Related information

Planning your large transfer

AWS OFFICIAL
AWS OFFICIALUpdated 2 years ago