How to aggregate records in S3 using IOT Rules and Action?

0

I'd like to aggregate prediction MQTT messages generated by the Greengrass ML Feedback Connector over a certain period into a JSON-Lines file in S3: Which may then be used as a SageMaker Ground Truth input manifest, or for other analysis.

I'm not clear from the doc on the IoT Rules S3 action whether:

  • It would be safe to have multiple (perhaps close/concurrent) messages pushing to the same S3 bucket & key via Rules
  • Messages would be concatenated into one file (with a separator?), or overwrite
  • There's enough flexibility in the IoT SQL functions to control the level of aggregation nicely (Seems like day/week/month would be tough, but probably could just do something with integer-dividing the timestamp() number

It seems like this is a use case that IoT Analytics has tools to handle, but I'm just trying to concatenate the messages together, not perform analyses/proper aggregations.

AWS
전문가
Alex_T
질문됨 4년 전649회 조회
1개 답변
0
수락된 답변

One option would be use Firehose to batch the data:

IoT Core --> Rule --> Firehose --> S3

In this flow, Firehose writes the messages in batches to objects stored in S3.

답변함 4년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인