- Più recenti
- Maggior numero di voti
- Maggior numero di commenti
You can think of the KCL as a micro-batch system. An instance of the KCL will take a micro-batch of records from a single shard in a Kinesis Stream and then one of two things happens: either it is successful in processing those records or it is not. If it is successful, it will move on and get the next micro-batch of records from the stream. If it is not successful, the system will attempt to re-process the same micro-batch.
If the KCL is interrupted for some reason (eg an EC2 scale event, EC2 failure etc), then it will not be successful in processing the current micro-batch. The micro-batch will therefore be re-processed by another EC2 instance that is running the KCL.
When you start the KCL, it creates a table in DynamoDB that tracks the state of the processing against each shard in the Kinesis Stream. This includes a lease system, so the overall system can know if an agent that is processing records from a shard has stopped responding and the batch needs to be reassigned to another agent.
Contenuto pertinente
- AWS UFFICIALEAggiornata 7 mesi fa
- AWS UFFICIALEAggiornata 10 mesi fa
- AWS UFFICIALEAggiornata 4 anni fa
- AWS UFFICIALEAggiornata un anno fa