- Newest
- Most votes
- Most comments
Hi ozy cozy,
you can use [IoT rule actions](ozy cozy) to store your messages in a variety of services for example, DynamoDB, Timestream or S3. What data store is the best option for you depends on your use-case. S3 as object store is good in term of pricing and you can query data with Athena. When you need to store the data in a database you can use as mentioned DynamoDB, Timestream, OpenSearch or RDS.
You only mention thousands of message but not in which interval you plan to ingest messages. The mentioned databases are usually built for high volume traffic. If they would match your use case in terms of the amount of data is something that you can find on the quota documentation for each service.
Cheers,
Philipp
Thanks Philipp, I am getting data very frequently. like every seccond data is streaming
Relevant content
- asked 5 years ago
AWS OFFICIALUpdated 2 years ago- AWS OFFICIALUpdated 6 months ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 2 years ago
Greetings! Philipp's response is spot-on. To add on, the requirement to display data on a screen should not be the primary factor in determining whether to use SQL or noSQL. Instead, consider the structure of the data being stored. If it is relatively static and known in advance, SQL may be a better fit. However, if the data structure is likely to vary, a noSQL may be a better option. Additionally, the type of data to be displayed, such as real-time monitoring or dashboards, should also be taken into consideration when selecting a technology stack. It's worth to check AWS IoT Analytics (https://aws.amazon.com/iot-analytics/) might cover your use cases.
Thanks Jordi, that was my question actually, I do have very structured data and comes from defined devices. Prefer to use RDS for that reason. Was trying to get some advice on that. If I use postgres DB for recent 30 days data, and keep everything else in S3 or Glacier will that be a good solution? Thanks
Hey Ozy, if your queries are sparse and might tolerate some response time, you might get rid of RDS altogether and use S3 + Athena (S3 has an automatic life-cycle you can leverage to push data to Glacier). If you do need RDS, I'd recommend to have all the data in S3 as well (storage is very cheap compared to RDS), and have a daily purge mechanism on RDS side for > 30 days (I don't think Postgres supports TTL natively). Cheers.
Hi Jordi, the plan is keep last 30 days in RDS, and also keep everything in s3 as you said. The other part data will be pushed to Dashboard realtime throught AppSync. Basicall flow: device -----> IOTCore ---->Rule-----Lambda--->AppSync(graphql)---broadcast and save(S3/DB).