How do I troubleshoot Kinesis Data Streams issues in DynamoDB?

Short description

The following are common issues that you might experience when you use Kinesis Data Streams with your DynamoDB table:

• Your Kinesis Data Streams records fail.
• Your costs are high or fluctuate.

To help you troubleshoot issues, review your AWS CloudTrail logs and monitor Kinesis Data Stream metrics in Amazon CloudWatch.

Note: The default storage duration for data in DynamoDB and Kinesis Stream is set to a 24-hour window. However, you can extend the retention period for your stream to preserve data for up to one year (365 days or 8,760 hours).

Resolution

Troubleshoot failed records 

To determine why a record fails, review your CloudTrail logs. The following are common reasons why a record might fail:

• You receive the AccessDeniedException error because you don't have the correct AWS Identity and Access Management (IAM) permissions. To resolve this issue, modify the permissions policy for your IAM role.
• You receive the KMSThrottlingException error when a request is denied because of throttling. To resolve this issue, see Why did I receive a ThrottlingExceptions error with API requests that are made to AWS KMS?
• You receive the ProvisionedThroughputExceededException error because the request rate for the stream is too high, or the requested data is too large for the available throughput. To resolve this issue, reduce the frequency or size of your requests.

If you get the ProvisionedThroughputExceededException error and the stream is in provisioned capacity mode, then you have an insufficient number of shards. Provision more shards to manage your table read and write operations.

If you get the ProvisionedThroughputExceededException error and the stream is in on-demand capacity mode, then you might be using an incorrect partition key. An incorrect partition key can unevenly distribute your data across shards. Uneven data distribution can overburden some shards with records that exceed the 2 MB per second read rate quota and up to 1 MB per second write rate quota. 

To resolve this issue, choose a partition key that evenly distributes data across your shards. 

Choose the right capacity mode to optimize costs

To optimize the performance and costs of your Kinesis data streams with DynamoDB, choose the correct capacity mode.

If your DynamoDB database workload significantly fluctuates and is difficult to predict, then select on-demand capacity mode. Because Kinesis Data Streams automatically manages the shards, you don't need to plan your capacity.

If your workload is more stable and predictable, then select provisioned capacity mode. You can specify the number of shards that are required to accommodate the change data capture records from your DynamoDB table. 

Use CloudTrail logs to see your API activity

To troubleshoot data streams with control plane API activities, review your CloudTrail logs. CloudTrail logs all control plane operations and includes stream activity information, such as when a stream is turned on or turned off. Analyze the CloudTrail logs for issues with provisioned throughput or Kinesis and DynamoDB resources. Then, use the Kinesis Data Streams API to manage your shards.

Use Kinesis Data Streams metrics to further troubleshoot

To monitor Kinesis Data Streams metrics, use CloudWatch.

The following metrics help you determine common issues:

• The GetRecords.IteratorAgeMilliseconds metric shows the age of the most recent record received from all GetRecords calls to a data stream for a specified time window.
• The GetRecords.Latency metric shows the duration of each GetRecords operation on a Kinesis stream for a specified time window.
• The ThrottledPutRecordCount metric shows the count of records that are throttled because of an insufficient capacity for the data stream.
• The WriteProvisionedThroughputExceeded and ReadProvisionedThroughputExceeded metrics show the number of records that are rejected because of throttling on the data stream for a specified time window.

Related information

Auto scaling Amazon Kinesis Data Streams using Amazon CloudWatch and AWS Lambda

Troubleshooting Kinesis Data Streams consumers