Kinesis Analytics for SQL Application Issue

0

Hello, I am having trouble to properly handle query with tumbling window. My application sends 15 sensor data messages per second to Kinesis Data Stream, which is used as an input stream for Kinesis Analytics application. I am trying to run an aggregation query using a GROUP BY clause to process rows in a tumbling window by 60 second interval. The output stream then sends data to a lambda function. I expect that the messages should arrive at lambda every 60 seconds but instead, they arrive much faster, almost every second, and the aggregations don't work as expected. Here is the CloudFormation template that I am using:

ApplicationCode: CREATE OR REPLACE STREAM "SENSORCALC_STREAM" (
"name" VARCHAR(16),
"facilityId" INTEGER, "processId" BIGINT, "sensorId" INTEGER NOT NULL, "min_value" REAL, "max_value" REAL, "stddev_value" REAL);

    CREATE OR REPLACE PUMP "SENSORCALC_STREAM_PUMP" AS 
    INSERT INTO "SENSORCALC_STREAM" 
    SELECT STREAM      
        "name", 
        "facilityId",             
        "processId",             
        "sensorId",        
        MIN("sensorData") AS "min_value",
        MAX("sensorData") AS "max_value",
        STDDEV_SAMP("sensorData") AS "stddev_value"
        
    FROM "SOURCE_SQL_STREAM_001"
    GROUP BY "facilityId","processId", "sensorId", "name",
         STEP("SOURCE_SQL_STREAM_001".ROWTIME BY INTERVAL '60' SECOND);

KinesisAnalyticsSensorApplicationOutput: Type: "AWS::KinesisAnalytics::ApplicationOutput" DependsOn: KinesisAnalyticsSensorApplication Properties: ApplicationName: !Ref KinesisAnalyticsSensorApplication Output: Name: "SENSORCALC_STREAM" LambdaOutput: ResourceARN: !GetAtt SensorStatsFunction.Arn RoleARN: !GetAtt KinesisAnalyticsSensorRole.Arn DestinationSchema: RecordFormatType: "JSON"

I would really appreciate your help in pointing what I am missing. Thank you, Serge

Serge
질문됨 2년 전104회 조회
답변 없음

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠