Hi Team,
Our Question:
Is there a feature that offered by Kinesis that allows us to enrich records in the same Kinesis stream? (get the plain record -> enrich it -> put it back into the same Kinesis, or enrich the records in place without take it out of the stream).
Background:
We are try to come up with a solution to enrich data in Kinesis Stream by join the incoming plain data with some static data.
After research I see there are many ways to do so, such as use Flink, Lambda, or EventBridge Pipes.
However, Flink is not an option for us. We use Flink to do aggregation task already, we don't want to centralize all tasks in a same service, plus it may be costly and difficult to scale in our use case.
For Lambda and EventBridge, the problem is that we have to create a new Kinesis stream to deliver the enriched data to Flink.
Is there any way to enrich the records in place without create a new Kinesis stream?