How to connect Kinesis Data Streams with EMR in EMR Studio?
0
I need to load data from Kinesis Data Streams to EMR via EMR Studio. I Follow this sample but doesn't work: https://github.com/awslabs/spark-sql-kinesis-connector
질문됨 3달 전1129회 조회
1개 답변
관련 콘텐츠
AWS 공식업데이트됨 3년 전- AWS 공식업데이트됨 6달 전
- AWS 공식업데이트됨 4년 전
Follow the error: An error was encountered: [STREAM_FAILED] Query [id = 8e197d37-61c3-48de-b388-b5b7e4b0078d, runId = 7a4bf9bb-97af-495e-8f61-edd54ef7d8dc] terminated with exception: Unable to fetch committed metadata from previous batch id 0. Some data may have been missed Traceback (most recent call last): File "/mnt1/yarn/usercache/livy/appcache/application_1709526661976_0019/container_1709526661976_0019_01_000001/pyspark.zip/pyspark/sql/streaming/query.py", line 221, in awaitTermination return self._jsq.awaitTermination() File "/mnt1/yarn/usercache/livy/appcache/application_1709526661976_0019/container_1709526661976_0019_01_000001/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1322, in call return_value = get_return_value( File "/mnt1/yarn/usercache/livy/appcache/application_1709526661976_0019/container_1709526661976_0019_01_000001/pyspark.zip/pyspark/errors/exceptions/captured.py", line 185, in deco raise converted from None pyspark.errors.exceptions.captured.StreamingQueryException: [STREAM_FAILED] Query [id = 8e197d37-61c3-48de-b388-b5b7e4b0078d, runId = 7a4bf9bb-97af-495e-8f61-edd54ef7d8dc] terminated with exception: Unable to fetch committed metadata from previous batch id 0. Some data may have been missed
also I follow this example: https://github.com/aws-samples/spark-kinesis-connector-emr-sample