- Newest
- Most votes
- Most comments
Hello
I understand that you would like to add the timestamp to the s3 objects written by the s3-sink connector.
The s3 sink connector writes with the ojects to s3 with the format "<topic>+<kafkaPartition>+<startOffset>.<format>", refer [1] for the details of the format.
You can modify the object name either by using lamdba function to read the messages in Kafka topics, refer[2] for further details Kinesis firehose can also be used to write the messages in kafka topic to s3 with required format, refer[3] for further details.
With regards to the error that you have mentioned, can you kindly open a case with AWS Support to investigate with Cluster, connector and IAM details.
References:
[1] https://docs.confluent.io/kafka-connectors/s3-sink/current/overview.html#s3-object-names/
[2] https://aws.amazon.com/blogs/compute/using-amazon-msk-as-an-event-source-for-aws-lambda/
Relevant content
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 2 years ago