Best Practise and cost effective way to stream data from Kafka topic to Redshift

Hello Britto,

Thanks for reaching out here. Amazon Redshift natively supports streaming ingestion from Amazon MSK to Redshift [1].Kafka is an open source platform and it is not directly integrated with Redshift streaming ingestion solution . Hence it is difficult for us to provide any recommendations or support as we do not posses the expertise and falls outside of our scope of support here. For information around the best practices related to it, I would request you to review the Kafka /3rd party integration forums [2][3][4] or GitHub [5][6]. Please note, these are 3rd party documentations are for your reference only.As part of AWS team, we are constrained to provide any support on the solution's mentioned on these docs. Additionally for specific solution-ing advise for integrating AWS technologies, you can also reach out to our Solutions Architect specialists team by filling out the form here -> https://aws.amazon.com/contact-us/sales-support-eks/ https://pages.awscloud.com/awsmp-contact-us.html

[1] https://docs.aws.amazon.com/redshift/latest/dg/materialized-view-streaming-ingestion.html. [2] https://estuary.dev/kafka-to-redshift/ [3] https://www.confluent.io/hub/confluentinc/kafka-connect-s3 [4] https://airbyte.com/how-to-sync/kafka-9-to-redshift-destination [5] https://github.com/confluentinc/kafka-connect-jdbc/issues/860 [6] https://github.com/lepfhty/kafka-connect-redshift