Document Compression in DocumentDB

0

We've been using DocumentDB for a few months and we are seeing a handful of outlier documents that are over the 16MB file size limit.
We've looked at a few different options for dealing with these documents, but we were curious if the 'new' document compression feature might help us insert these larger documents.

We have a few questions that we couldn't find the answers to:

  • Can we use LZ4 to compress documents before insertion? (benefits: smaller payloads over the network, possibility of allowing larger documents to be inserted)
  • How does the 16MB size limit apply to a compressed document? (does the uncompressed document size matter? Where is the size checked?)
  • Is there any information on how indexes, search speed, etc are impacted by compression?

Thanks!

Nick
質問済み 6ヶ月前258ビュー
1回答
0

Q. With DocumentDB compresssion, does the size limit apply pre-compression or post-compresssion?

Amazon DocumentDB only compresses documents with a size of 2KB and larger and compression is only supported on Amazon DocumentDB version 5.0 and higher.

[+] https://docs.aws.amazon.com/documentdb/latest/developerguide/doc-compression.html#compression-guidelines [+] https://docs.aws.amazon.com/documentdb/latest/developerguide/doc-compression.html#enabling-compression

Q. Can documents be 'pre-compressed' before insertion?

Yes, you can enable document compression at a collection level and view compression metrics as needed by measuring the storage gains through compression metrics such as storage size of compressed documents and compression status. Please be advised that as per our documentation, document compression is disabled by default and this setting is currently not modifiable. As a result, you may consider enabling document compression while creating your collections to be migrated on the target Amazon DocumentDB cluster before starting the document insertion/migration.

[+] https://docs.aws.amazon.com/documentdb/latest/developerguide/doc-compression.html

[+] https://docs.aws.amazon.com/documentdb/latest/developerguide/doc-compression.html#manage-existing-collections

Q. Is there any information on how indexes, search speed, etc are impacted by compression?

Compressed documents in Amazon DocumentDB are up to 7x smaller than uncompressed documents. Compressed documents require less storage space and IO operations during database reads and writes, leading to lower storage and IO costs.

[+] https://aws.amazon.com/about-aws/whats-new/2023/07/amazon-document-db-mongo-db-document-compression/

AWS
Nandan
回答済み 6ヶ月前
  • Thanks for the response Nandan!
    I edited my question to clarify some of my questions. When I said size limit, I was referring to the 16MB document size limit. I have looked at the documents you are linking before I posted and, unfortunately, they don't have the specific answers to the questions I was asking.

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ