Changing the encryption type while reading data from shards of Kinesis Stream

Question

Hello

I am using KMS key to encrypt a kinesis stream. I have two separate threads in one lambda. One of the lambda is posting data into streams by using CMK encryption  type and the other thread waits a while and starts reading the data inside lambda . The thing is while reading the data it changes the type of the encryption key of the current stream and uses AWS Managed Key instead of CMK Key and finalises reading the data.

Reading the stream process terminates successfully. I am making my implementation at CodeEditor of AWS by using Python programming language. I am using Boto3 implementation from (https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html)

Everything is fine. Nothing is crashed and all the writing to stream and reading from the stream are done successfully. But I have following questions :

A- )  When the encryption type changes while reading the stream   what happens? Such like that I have 1000 data in one stream and I read 100 data from the stream and afterwards immediately I changed the encryption type from CMK to AWS Managed Key while still reading the remaining data. Following 900 data will be AWS Managed Key and the other 100 data that I read before changing the encryption type still remains as Custom Managed Key KMS instead of AWS Managed Key ?

B-) When I change the encryption type with the following method

client.start_stream_encryption(
      StreamName='kms-investigation-kinesis-stream',
      EncryptionType='KMS',
      KeyId=''
    )

then what happens ? Encryption type of unread items will be changed or encryption type of stream will be changed? Could you please enlighten me which one will be changed ?

C-) What happens if I delete CMK key ? Such like I read 101 key with the Custom Managed KEY and then I changed the encryption type and I read 900  key.  When all reading process is completed I deleted CMK key,  what happens with the reamining 101 key ? They will be deleted  as well ? Because as you know that 101 Key was encrypted and read by CMK key and afterwards encryption type has been changed. The remaining 101 will be gone or they will be encrypted as AWS Managed Key instead of Custom Managed Key?

Many thanks for your helps.

Best Regards
Alper

Yossi Cohen · Answer

Hi Alper,

[AWS Kinesis documentation](https://docs.aws.amazon.com/streams/latest/dev/what-is-sse.html) says:
> Server-side encryption is a feature in Amazon Kinesis Data Streams that automatically encrypts data before it's at rest by using an AWS KMS customer master key (CMK) you specify. Data is encrypted before it's written to the Kinesis stream storage layer, and decrypted after it’s retrieved from storage. As a result, your data is encrypted at rest within the Kinesis Data Streams service...

Based on my experience, when you change the encryption settings on an AWS Kinesis stream, the new encryption configuration only applies to records that are ingested after the change. AWS Kinesis does not retroactively encrypt records that were already in the stream before the change.

A) Once you update the KMS key, all subsequent records will be encrypted using the new KMS configuration before they are stored in Kinesis.

B) Only records that are added to the stream after the KMS key update will be encrypted with the new settings.

C) If you delete a KMS key (especially if you scheduled and confirmed its deletion), any data encrypted with that key will become unrecoverable.

For more information about the Kinesis API and its error handling, you can visit [this link](https://docs.aws.amazon.com/kinesis/latest/APIReference/API_GetRecords.html).

Aman_A · Answer

Hello Alper,

As per my research the new encryption configuration gets applied to records that come after the settings is applied.

Changing the encryption type while reading data from shards of Kinesis Stream

관련 콘텐츠