How to enable Kinesis Firehose new line delimiter via CloudFormation?

Question

There is a relatively new feature for Firehose to append new line at the end of records before saving them to S3: https://docs.aws.amazon.com/firehose/latest/dev/create-destination.html

by enabling "new line delimiter" option. I can see this option in the console, but I cannot see anything mentioned about it in the KinesisFirehose CF documentations:
* https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-kinesisfirehose-deliverystream-extendeds3destinationconfiguration.html
* https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-kinesisfirehose-deliverystream.html#cfn-kinesisfirehose-deliverystream-extendeds3destinationconfiguration

Is it possible to enable the "new line delimiter" option via CloudFormation? If yes, then how?

Matthew-Gu · Accepted Answer

The new line delimiter configuration is a type under the Processing Configuration within your Cloud Formation Template for your S3 Destination.

For JSON this will look like:

{
                    "Type": "AppendDelimiterToRecord",
                    "Parameters": [
                        {
                        "ParameterName": "Delimiter",
                        "ParameterValue": "\n"
                      }
                    ]

For the YAML this will look like:

- Type: AppendDelimiterToRecord
            Parameters:
            - ParameterName: Delimiter
              ParameterValue: "\n"

For a more detailed example of how this should look from a higher level within your template, please refer to the documentation [1] below:

[1] AWS::KinesisFirehose::DeliveryStream - Specify an Amazon S3 Destination for the Delivery Stream - https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-kinesisfirehose-deliverystream.html#aws-resource-kinesisfirehose-deliverystream--examples--Specify_an_Amazon_S3_Destination_for_the_Delivery_Stream

For a list of all Processor type configurations, please refer to the documentation [2] below:

[2] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-kinesisfirehose-deliverystream-processor.html#:~:text=AppendDelimiterToRecord

How to enable Kinesis Firehose new line delimiter via CloudFormation?

Relevant content