Multiple customer-defined partition keys in Timestream

Hello Drew,

In Amazon Timestream, the partitioning scheme is indeed designed to handle complex partitioning logic, but the distinction between how many partition keys you can have might seem confusing based on the user interface and documentation.

As of the latest information available to me, Timestream allows you to specify multiple attributes as part of a single customer-defined partition key (often referred to as a "composite" partition key). This means that while the UI and API expect a singular "partition key" input, this key can itself be composed of multiple attributes from your data.

Here's how it works:

Composite Customer-Defined Partition Keys: When setting up a customer-defined partition key, you can include multiple columns or attributes within this key. For example, if you have data attributes like region and device_id, you can create a composite key that includes both. This is useful for query optimization and data management, as it allows you to tailor the partitioning of your data more closely to your query patterns. UI Limitation: If the Timestream management console (UI) only allows you to add a single attribute to the partition key field, this might be a limitation or a design choice in the UI itself. To specify multiple attributes in the partition key, you might need to use the AWS CLI or SDKs. Using CLI/SDKs: You can specify multiple attributes for your partition key when using the AWS Command Line Interface (CLI) or one of the AWS SDKs. Here’s a simplified example of how you might specify a composite key using the AWS CLI:

aws timestream-write create-table --database-name YourDatabaseName --table-name YourTableName --retention-properties '{"MemoryStoreRetentionPeriodInHours": "24", "MagneticStoreRetentionPeriodInDays": "365"}' --customer-defined-partition-key-attributes '[{"AttributeName": "region", "AttributeType": "VARCHAR"}, {"AttributeName": "device_id", "AttributeType": "VARCHAR"}]'

This command sets up a table with a composite partition key consisting of region and device_id. Documentation: It seems like there might be a gap or an unclear explanation in the documentation if it suggests adding multiple keys in a way that the UI doesn't support directly. The documentation's mention of a list of partition keys is indeed about creating a hierarchy of attributes within a single composite key. If you are unable to add multiple attributes through the UI, I recommend trying the CLI approach as shown above, which provides more flexibility. If you encounter issues or discrepancies, reaching out to AWS support or providing feedback on the documentation might also help clarify the situation or lead to improvements in how these features are presented in the UI.

Feel free to explore these options, and hopefully, this clears up the confusion and helps you set up your Timestream tables as needed!