Getting botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found in BOTO3 while trying to create a s3 object

Question

I have a list of s3 Objects .i want to get the count of keys in Glacier and archive. Here is my code

```
s3 = boto3.resource(service_name='s3', aws_access_key_id=accesskey, aws_secret_access_key=secretkey)
count = 0
# latest object is a list of s3 keys
for obj in latest_objects:
    try:
        response = s3.Object(Bucket, obj)
        if response.storage_class in ['GLACIER', 'DEEP_ARCHIVE']:
            count=count+1
            print("To be restored: " + obj)
   except Exception as e:
       print(str(e))
       break
```
when I run this code I am getting botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found for some s3 keys. when I verified these keys are present in bucket. Not sure what's wrong, Again this happens only for some of the keys for the rest of the keys response = s3.Object(Bucket, obj) works fine

Any help is much appreciated

Anirban Chakraborty · Answer

When you create a folder via s3 console, it creates an object with the name appended by suffix "/" and that object is displayed as a folder in the s3 console. So it will be returned on a HeadObject operation.

But if you create the file test1/test2/file.txt that's just the name of the file, it doesn't create test1/ and test2/ for you. Object storage is a flat file structure unlike block storage.

When you create a folder from the console it mentions that behavior as well:
![Guide to solve the querry](/media/postImages/original/IMPxbVQHdVQKmRQFcaPknhYQ)

Getting botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found in BOTO3 while trying to create a s3 object

Relevant content