By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Document Compression in DocumentDB

0

We've been using DocumentDB for a few months and we are seeing a handful of outlier documents that are over the 16MB file size limit.
We've looked at a few different options for dealing with these documents, but we were curious if the 'new' document compression feature might help us insert these larger documents.

We have a few questions that we couldn't find the answers to:

  • Can we use LZ4 to compress documents before insertion? (benefits: smaller payloads over the network, possibility of allowing larger documents to be inserted)
  • How does the 16MB size limit apply to a compressed document? (does the uncompressed document size matter? Where is the size checked?)
  • Is there any information on how indexes, search speed, etc are impacted by compression?

Thanks!

Topics
Database
Tags
Amazon DocumentDB
Language
English
Nick
asked 6 months ago247 views
1 Answer
0

Q. With DocumentDB compresssion, does the size limit apply pre-compression or post-compresssion?

Amazon DocumentDB only compresses documents with a size of 2KB and larger and compression is only supported on Amazon DocumentDB version 5.0 and higher.

[+] https://docs.aws.amazon.com/documentdb/latest/developerguide/doc-compression.html#compression-guidelines [+] https://docs.aws.amazon.com/documentdb/latest/developerguide/doc-compression.html#enabling-compression

Q. Can documents be 'pre-compressed' before insertion?

Yes, you can enable document compression at a collection level and view compression metrics as needed by measuring the storage gains through compression metrics such as storage size of compressed documents and compression status. Please be advised that as per our documentation, document compression is disabled by default and this setting is currently not modifiable. As a result, you may consider enabling document compression while creating your collections to be migrated on the target Amazon DocumentDB cluster before starting the document insertion/migration.

[+] https://docs.aws.amazon.com/documentdb/latest/developerguide/doc-compression.html

[+] https://docs.aws.amazon.com/documentdb/latest/developerguide/doc-compression.html#manage-existing-collections

Q. Is there any information on how indexes, search speed, etc are impacted by compression?

Compressed documents in Amazon DocumentDB are up to 7x smaller than uncompressed documents. Compressed documents require less storage space and IO operations during database reads and writes, leading to lower storage and IO costs.

[+] https://aws.amazon.com/about-aws/whats-new/2023/07/amazon-document-db-mongo-db-document-compression/

AWS
Nandan
answered 6 months ago
  • Nick
    6 months ago

    Thanks for the response Nandan!
    I edited my question to clarify some of my questions. When I said size limit, I was referring to the 16MB document size limit. I have looked at the documents you are linking before I posted and, unfortunately, they don't have the specific answers to the questions I was asking.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content