Setting Kinesis for API

Question

Hi,

to extract data from an application (SaaS), we are creating an API and through API we ingest into Kinesis and than process it over Databricks before final consumption.
I need to know what are the steps i need to follow to create Kinesis stream to pull data from the API

Answer

Hi,

This SumoLogic post is a good article to understand Kinesis globally: https://www.sumologic.com/blog/aws-kinesis-streams-getting-started/

This RedHat post gives all hands-on details to create a Kinesis stream from a producer to 1 or N consumers: https://developers.redhat.com/articles/2022/02/09/create-data-stream-amazon-kinesis

In your case, the consumer will use DataBricks api to import. This video re. Kinesis best practices may be useful to design your solution: https://www.youtube.com/watch?v=UE34CWAhT3o

Best,

Didier

Setting Kinesis for API

Relevant content