Simulating GPS sensors sending data to AWS Kinesis

Question

Hi everyone,

I am working on a project related to mobility data, and I thought you could help me with an issue I have : I would need something to simulate IoT GPS sensors for the testing phase of the project. I currently use batches of daily mobility data (‘dummy’ data found on internet), but I would like to use Kinesis Firehose to gather data coming from GPS sensors and produce those daily batches. I would like to build an infrastructure capable of collecting gps data coming from cars (and produce a dashboard to analyse mobility data).

Maybe some of you have already had this kind of problem before, when you need to test a solution for GPS data without using real data directly.

Do you know such a way (using AWS tools, or others) to send GPS data to Kinesis firehose from different sources simultaneously ?

Thanks for your consideration,

Sami

Answer

if you open to use some opensource tools like fluentbit - it is relatively straightforward to generate some random data and push it to kinesis firehose

https://docs.fluentbit.io/manual/pipeline/inputs/dummy
 
https://docs.fluentbit.io/manual/pipeline/inputs/random

https://docs.fluentbit.io/manual/pipeline/outputs/firehose

Answer

Amazon Kinesis Data Firehose is a good fit to ingest data and process them as batches, enrich and store them on Amazon S3.

You can use Amazon Kinesis Data Streams to feed data to Firehose. For your device data ingestion simulation use case; you can use AWS SDK in your preferred programming language to call Kinesis Streams **PutRecord API** ( https://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecord.html ) on your simulation data creator script. Thus, your data will be on Kinesis Streams, then you can configure it to be consumed by Firehose.

When you need to collect data from real devices, you can use AWS IoT Core for the entry point for your data. Your devices can have the AWS IoT Device SDK in your preferred programming language to publish data to AWS IoT Core data plane. AWS IoT Core offers **Rules** feature to forward data to AWS services. So you can configure **Kinesis Data Streams Rule** to forward incoming data to Amazon Kinesis Data Streams. In this case, **AWS IoT Core service** will be the place where Kinesis PutRecord API is called, instead of your custom script on the simuation use-case.

Simulating GPS sensors sending data to AWS Kinesis

Relevant content