How can I copy a list of specific files from one bucket directly to another bucket keeping the same folder structure in the target bucket, using aws CLI, without having to download the files first

Here's how you can do it:

Ensure those files are listed in a text file where each line contains the S3 path of one file to be copied. For example, your list might look like this:

s3://source-bucket/file1.txt
s3://source-bucket/innerfolder/file2.txt

Then, you'll need to use a script to read through your list of files and use the AWS CLI to copy each one.

#!/bin/bash

INPUT_LIST=path_to_your_list/list.txt
TARGET_BUCKET=target-bucket

while IFS= read -r line
do
    FILE_PATH=$(echo "$line" | sed 's/s3:\/\/[^\/]*\///')

    aws s3 cp "$line" "s3://$TARGET_BUCKET/$FILE_PATH"
done < "$INPUT_LIST"

This script reads each line from your list, extracts the file path excluding the bucket name, and then copies the file to the target bucket, preserving the folder structure.

Then, need to make the script executable, and execute the script:

chmod +x path_to_your_script/script.sh
./path_to_your_script/script.sh

If this has answered your question or was helpful, accepting the answer would be greatly appreciated. Thank you!