Questions tagged with Amazon Kinesis Data Streams

Amazon Kinesis Data Streams is a serverless streaming data service that makes it easy to capture, process, and store data streams at any scale.

Content language: English

Select tags to filter
Sort by most recent

Browse through the questions and answers listed below or filter and sort to narrow down your results.

120 results
I am using AWS Database Migration Service (DMS) to replicate Change Data Capture (CDC) data from Aurora PostgreSQL Serverless to Amazon Kinesis Data Streams. My Aurora PostgreSQL source endpoint works...
1
answers
0
votes
27
views
asked 3 days ago
My previous post: [https://repost.aws/questions/QUj4XJYpDOSIKftobhiOsNpQ#ANzFBcnc_-SHCxxAuNMWPvIw ]() We're building a custom Node.js consumer for Kinesis Data Streams and intentionally avoiding KCL d...
1
answers
0
votes
42
views
asked a month ago
I'm working on a Node.js application that polls from AWS Kinesis and pushes to Event Hubs. We have multiple poller instances where each poller needs to handle certain shards. Currently, we distribute ...
2
answers
0
votes
37
views
asked a month ago
For our Kinesis Data Streams setup, we're observing automatic retries on throttling errors, specifically `ProvisionedThroughputExceededException`. We have our own exponential backoff mechanism in plac...
1
answers
0
votes
105
views
asked a month ago
What are the main differences between a Data Lake and a Data Warehouse in AWS? I am working on a project and would like guidance on how to decide which one is better suited for specific use cases and ...
2
answers
1
votes
55
views
profile picture
asked a month ago
I am currently using AWS Lambda as a consumer for my Kinesis Data Streams. Here are the details of my setup: - **Kinesis Stream Configuration:** - **Number of Shards:** 8 - **Event Source Setting...
3
answers
0
votes
106
views
asked 2 months ago
We have some IoT devices using AWS IoT Core that send data every 15 min to a lambda that deserializes and uploads to a Timestream database. I recently came across Kinesis Data Streams and it sounds l...
2
answers
0
votes
105
views
asked 3 months ago
Description: We're seeking clarification on the behavior of GetIteratorAge metric in Kinesis Data Streams (KDS), as we're observing what appears to be unexpected behavior with iterator aging. Current...
1
answers
0
votes
94
views
asked 3 months ago
Description: We are experiencing consistent throttling with our Kinesis Data Stream (KDS) and seeking guidance on the root cause and resolution. Our key observations suggest the throttling behavior mi...
1
answers
0
votes
84
views
asked 3 months ago
I have a simple python code of Flink. I am consuming the data from the kafka and then sinking it to opensearch. I have used state and timers for this which is working fine in my local machine. I am ...
1
answers
0
votes
153
views
asked 3 months ago
We have requirement to transfer data from Snowflake to Redshift in Real time. The details are as follows, - Snowflake will be sending the data in real time for the required views. - These need to be p...
2
answers
0
votes
135
views
asked 4 months ago
Hi, My product is multi-tenant and each tenant is having isolated AWS RDS PostgreSQL DB. I have over 2000 tenants i.e. 2000 RDS dbs and over a period it will grow continuously. My product is kind o...
2
answers
0
votes
157
views
asked 5 months ago