How do I optimize performance when I use AWS CLI to upload large files to Amazon S3?

Short description

If you upload large files to Amazon S3, then it's a best practice to use multipart uploads. Also, configure an object size threshold for multipart uploads. When you use AWS CLI to upload files, all high-level aws s3 commands automatically perform a multipart upload when the object size exceeds the threshold. These high-level commands include aws s3 cp and aws s3 sync.

To optimize performance, choose one of the following methods:

• Use the AWS Common Runtime (CRT)-based client and customize the multipart upload configurations. Use this method for large-scale data transfers or complex workloads.
• Activate Amazon S3 Transfer Acceleration. Use this method for geographically distant transfers.

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

Option 1: Use the CRT-based client with the AWS CLI

To use the CRT-based client, complete the following steps:

1. Add the preferred_transfer_client to your AWS CLI config file:

   [default]s3 =
   preferred_transfer_client = crt

   Or, run the configure command:

   aws configure set default.s3.preferred_transfer_client crt

2. Run the cp command to verify that the CRT client is active:

   aws s3 cp large-file.zip s3://your-bucket-name/ --debug 2>&1 | grep -i crt

   Example output when the CRT client is active:

   aws s3 cp large-file.zip s3://your-bucket-name/ --debug 2>&1 | grep -i crt  
   \[DEBUG\] \[2025-12-01T07:58:46Z\] \[00007fe20f9b9340\] \[tls-handler\] - Set security policy to AWS-CRT-SDK-TLSv1.0-2025-PQ (minimum\_tls\_version: 128; cipher\_pref: 0)  
   2025-12-01 07:58:46,742 - MainThread - s3transfer.crt - DEBUG - Recommended CRT throughput target in gbps: None  
   2025-12-01 07:58:46,742 - MainThread - s3transfer.crt - DEBUG - Using CRT throughput target in gbps: 10.0

3. (Optional) Set the target_bandwidth to control the maximum throughput. Add a target_bandwidth value to your AWS CLI config file:

   \[default\]  
   s3 =  
       preferred\_transfer\_client = crt  
       target\_bandwidth = 1Gb/s

   Or, run the configure command:

   aws configure set default.s3.target_bandwidth 1Gb/s

   Note:  Configure this only if you have specific bandwidth requirements. AWS CLI automatically optimizes bandwidth based on your system capabilities. Misconfiguration of this value can overwhelm your network or system resources. It's a best practice to leave this option null.

After you activate the CRT-based client, AWS CLI automatically uses the client for file uploads. This improves performance and reliability compared to the standard AWS CLI, especially for large file uploads.

Customize multipart uploads with these parameters:

Note: Check that your machine has enough resources to support the maximum number of concurrent requests that you require.

Option 2: Activate S3 Transfer Acceleration

Amazon S3 Transfer Acceleration provides fast and secure transfers over long distances between your client and Amazon S3. Transfer Acceleration uses the globally distributed edge locations of Amazon CloudFront.

Transfer Acceleration incurs additional charges, so be sure that you review pricing. To determine if Transfer Acceleration improves the transfer speeds for your use case, review the Amazon S3 Transfer Acceleration Speed Comparison tool.

Note: Transfer Acceleration doesn't support CopyObject copies across AWS Regions.

To activate Amazon S3 Transfer Acceleration, complete the following steps:

1. Turn on S3 Transfer Acceleration for your bucket.

2. Specify the accelerated endpoint for a single file transfer:

   aws s3 cp large-file.zip s3://your-bucket-name/ --endpoint-url https://s3-accelerate.amazonaws.com

   Or, configure the endpoint in your CLI config file to apply to all file transfers:

   \[default\]  
   s3 =  
       use\_accelerate\_endpoint = true

Related information

AWS CLI S3 Configuration

What is the AWS Command Line Interface?