Detailed documentation on Amazon Glacier Select
IHAC looking to archive a fair amount of structured audit trail for clinical data which will be accessed infrequently in AWS S3 Glacier Deep Archive to be cost effective. A very small subset of records from this archived data will need to be accessed on-demand and Glacier Select seems like an efficient option to just restore the records needed.
Whereas there is good documentation on S3 Select, it is hard to find much on Glacier Select. Could somebody provide pointers to the detailed documentation including some code examples for the AWS SDK.
- Newest
- Most votes
- Most comments
Hi there,
Throwing some of the documentation here related to your ask:
- https://aws.amazon.com/blogs/aws/s3-glacier-select/
- https://docs.amazonaws.cn/en_us/amazonglacier/latest/dev/using-aws-sdk.html
- https://pages.awscloud.com/rs/112-TZM-766/images/2018_0317-STG_Slide-Deck.pdf
- https://docs.amazonaws.cn/en_us/sdk-for-javascript/v2/developer-guide/glacier-examples.html
Happy to help further if required.
Abhishek
I believe you'd have to use this API to restore the object from the offline storage class, but you could specify the
SelectParameters input construct to specify your query: https://docs.aws.amazon.com/AmazonS3/latest/API/API_RestoreObject.html#AmazonS3-RestoreObject-request-SelectParameters
Here's the equivalent CLI command, also accepting SelectParameters as input to the --restore-request parameter: https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/restore-object.html
The S3 pricing page https://aws.amazon.com/s3/pricing/ shows prices on the "Transform & query" tab for Select scans for all the online storage classes except Express One Zone, as well as for Glacier Flexible Retrieval as an offline/near-line class. For Deep Archive, it shows "n/a", so I wouldn't be surprised if restore requests using Select were refused. It's probably simplest to test by putting a couple of small, inexpensive test objects in Glacier Flexible Retrieval and Glacier Deep Archive and trying to run a restore with a select query.
Relevant content
- Accepted Answerasked 6 years ago
AWS OFFICIALUpdated 4 years ago
Note that of those documents, only the 2018 slide deck appears to discuss the Glacier storage classes in Amazon S3. The other documents are about Amazon S3 Glacier, which is a separate, legacy service no longer recommended for use (https://docs.aws.amazon.com/amazonglacier/latest/dev/introduction.html). Judging by the question mentioning the Deep Archive storage class, which is exclusive to the regular Amazon S3 service -- and not available in Amazon S3 Glacier -- I'd suggest paying particular attention to this distinction when reviewing the documents.