In CloudFormation (specifically CDK) how do I set Timestream's partition key strategy to default partitioning?

0

CfnTableProps doesn't seem to have a way to set the partitioning strategy to automatic, but it must be done at table creation time. In the documentation HERE it is under Schema.

If you create a table using the console, you get these options:

Custom partitioning - This is useful when the primary access pattern of the queries is filtering over a high cardinality dimension or measure. Default partitioning - This is useful when the primary access pattern of queries is measure based filtering. The values of 'measure_name' attribute is used as the partition key to partition the data.

I don't have a measurement name, my plan is to have several dimensions - "Customer" "Site" "System" and "Machine". I probably should just leave the partitioning to the default, but since it's a setting in the console I was perplexed why it's not in the Cfn config object. Is it okay to leave it set to Custom partitioning ... I want to leave it up to Timestream to decide how to partition, and I plan on providing it with a set of keys. If it helps, I could combine those into a single key kind of how you form DynamoDB primary keys, but if I just leave this the default the way CDK partitioned it, can I just set my four custom dimensions and Timestream will do the right thing in terms of automatically partitioning as it needs to?

partitioning key default

2 Answers
1

Hi, CloudFormation allows to define same fields that you point to with the link https://docs.aws.amazon.com/timestream/latest/developerguide/API_CreateTable.html

Look at https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-timestream-table-partitionkey.html You'll find exact same fields here.

Best,

Didier

profile pictureAWS
EXPERT
answered 9 months ago
0
Accepted Answer

After some digging and thinking about what Didier said I figured out how to solve this. Ultimately, the problem is not only that CDK itself doesn't support this but neither do the underlying Cloudformation libraries. In other words, you can't use even an L1 construct because the property doesn't exist in CfnTableProps yet. My guess is that it will soon (it's not very difficult) but it will take some time for this to roll up into a release.

However, this seems to work. Use an L1 construct to create your table:

let table = new timestream.CfnTable(stack,
        ID.TSDB_ALARMS_TABLE_ID,
        {
            databaseName: ID.TSDB_DB_NAME,
            tableName: ID.TSDB_ALARMS_TABLE_NAME,

            // the properties below are optional
            magneticStoreWriteProperties: {
                "EnableMagneticStoreWrites": true,
                "MagneticStoreRejectedDataLocation": {
                    "S3Configuration": {
                        "BucketName": ID.S3_TSDB_REJECT_BUCKET_NAME,
                        "ObjectKeyPrefix": "alarm_",
                        EncryptionOption: "SSE_S3"
                    }
                },
            },
            retentionProperties: {
                "MemoryStoreRetentionPeriodInHours": "24",
                "MagneticStoreRetentionPeriodInDays": "36525" /* 100 years */
            },
            tags: [],
        });

then, do this to it:

    table.addPropertyOverride("Schema", {
        "CompositePartitionKey": [
            {"Name": "pk", "Type": "DIMENSION", EnforcementInRecord: "REQUIRED"}
        ]
    });

I think this is the short term solution: use addPropertyOverride() and you don't have to mess around with a dangerous escape hatch where you have to directly muck around with the DOM nodes.

Leaving this here on repost just in case someone else has the same question.

profile picture
wz2b
answered 9 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions