All Content tagged with Amazon Managed Streaming for Apache Kafka (Amazon MSK)
Amazon MSK makes it easy to ingest and process streaming data in real time with fully managed Apache Kafka.
Content language: English
Filter content
Select tags to filter
Sort by
Sort by most recent
402 results
Hello,
I am unable to create any MSK Connect connector in account 475820189572 (region us-east-2). The API consistently fails after parameter validation, and the console masks the real error as a per...
Hi AWS Team,
We are currently testing Amazon MSK with an S3 Sink Connector. The MSK cluster, EC2 instance, custom connector plugin, and required IAM roles have been configured.
However, we are unabl...
Jeongho KimSUPPORT ENGINEER
published 6 days ago0 votes49 views
When your security policy prevents security group references and you allow each Amazon MSK broker private IP individually in security groups or firewalls, it is hard to tell which IPs stay and which c...
Hyangmok BaekSUPPORT ENGINEER
published 17 days ago0 votes73 views
n MSK Connect restart preserves all configuration and committed offsets, runs asynchronously, and requires the connector to be RUNNING with no other lifecycle operation in progress. This article cover...
Hyangmok BaekSUPPORT ENGINEER
published 17 days ago0 votes63 views
On Amazon MSK, a Kafka Streams application can rebalance at a regular interval even though no instance joined or left. These are usually probing rebalances.
This article covers where to get the client...
Hyangmok BaekSUPPORT ENGINEER
published 17 days ago0 votes74 views
With processing.guarantee=exactly_once_v2, Kafka Streams overrides the producer transaction.timeout.ms to 10,000 ms (not the documented 60,000 ms) and commit.interval.ms to 100 ms. This article shows ...
Hyangmok BaekSUPPORT ENGINEER
published a month ago0 votes114 views
When a Kafka producer fails with TimeoutException: Topic ... not present in metadata after 60000 ms, most people check whether the topic exists or whether the network is broken. But this error also oc...
I have an EventBridge Pipe (`my-pipe`) using an MSK Serverless cluster (`my-msk-cluster`, ACTIVE) as a source, targeting a CloudWatch Logs group. It's stuck in this state:
```
PROBLEM: Pipe VPC event...
Masudur Rahaman SayemEXPERT
published 2 months ago0 votes92 views
This post shows how to authenticate a Kafka client to Amazon MSK using a Google OAuth/OIDC identity, with no long-lived AWS keys.
Hyangmok BaekSUPPORT ENGINEER
published 2 months ago0 votes105 views
Cluster sizing for MSK Express brokers is usually calculated from the official per-broker throughput quotas. But workloads that use Kafka transactions hit a different bottleneck - commit processing - ...
Hyangmok BaekSUPPORT ENGINEER
published 2 months ago0 votes175 views
Express brokers offer up to 3x throughput per broker, 20x faster scaling, and no storage management. But if you operate them the way you operate Standard brokers, you will run into unexpected behavior...
Hyangmok BaekSUPPORT ENGINEER
published 2 months ago1 votes138 views
When consumer lag spikes, the usual reaction is to add consumers or partitions. But when lag piles up on a single partition, scaling out does not help and can even make things worse: the cause is not ...