Can boto3 display progress of copying from one s3 bucket to another?

Question

Hello,  
I have a script that uses boto3 to copy files from a backup glacier bucket to our production bucket. Was wondering if I could add something to the script that will display some type of progress indicator in my terminal window when I run the script? Whether it's a loading bar or "x of x GB (speed)" like when you use AWSCLI to do a cp action, it'd be nice to have something to estimate how long the copy will take. Below is the part of the script that performs the copy:

```
for line in prefixes:
    myPrefix = line.rstrip()
    print("Looking at prefix "+myPrefix)
    for objectSummary in bucket.objects.filter(Prefix=myPrefix):
        key = objectSummary.key
        if key.endswith("/"):
          continue
        print("Looking at key "+key)
        copy_source = {'Bucket': sourceBucket, 'Key':key}
        s3.meta.client.copy(CopySource = copy_source, Bucket = destinationBucket, Key = key, Config = transferConfig)
```

Answer

A coworker of mine came up with a solution, had to create a custom class, some code snippets below for anyone who might want to implement similar functionality...

```
import boto3
import sys

class S3CopyProgress:
  def __init__(self, objectSummary):
    self.objectSummary = objectSummary
    self.bytesTransferredTotal = 0

  def updateProgress(self, bytesTransferred):
    self.bytesTransferredTotal += bytesTransferred
    percentComplete = round((self.bytesTransferredTotal/self.objectSummary.size)*100)
    sys.stdout.write('[%s] %s%s of %s\r' % (self.objectSummary.key, percentComplete, '%', self.objectSummary.size))
    sys.stdout.flush()
```

```
for line in prefixes:
    myPrefix = line.rstrip()
    print("*** Looking at prefix "+myPrefix+" ***")
    objs = bucket.objects.filter(Prefix=myPrefix)
    for objectSummary in objs:
        key = objectSummary.key
        if key.endswith("/"):
          continue
        progressMonitor = S3CopyProgress(objectSummary)
        copy_source = {'Bucket': sourceBucket, 'Key':key}
        s3.meta.client.copy(CopySource = copy_source, Bucket = destinationBucket, Key = key, Config = transferConfig, Callback = progressMonitor.updateProgress)
        print("")
```

Can boto3 display progress of copying from one s3 bucket to another?

Relevanter Inhalt