Designing scalable IoT systems on AWS allows companies to focus on core business requirements without worrying about the infrastructure management. It’s important to remember that scalability is important, but also keeping systems cost efficient are also a key concern when designing proper IoT systems. Batching data for processing, using appropriate services for different types of data, and storing that data appropriately will allow any IoT system to cost effectively scale.
Using IoT Core
The main benefit of using AWS’s IoT Core is to allow two-way communication to your connected devices. Using MQTT protocol, your devices can transmit data, but also have commands transmitted to them. Some IoT systems do not require this functionality, so if your devices only need to transmit data then IoT Core is not necessary, but could still be useful.
Also, the limitations of IoT Core MQTT payload (128kB max per request) means the data flow design is critical when using IoT Core.
Scale with Data
Assigning a single IoT Core topic to manage all incoming data may be a simple solution to easily set up an IoT cloud service, however, this can be a poor choice for scalability. Depending on the amount of data that will be coming through the topic, the service can be flooded and experience event failures.
IoT systems should be able to handle almost real-time, high-volume, streamed data from devices and gateways. The design of the infrastructure must be able to scale with the volume of data and devices that will be connected now and in the future.
The most efficient way to efficiently achieve near real-time data exchange while making sure not to overload the database processing, is to batch the data into a message queue, load it into a memory cache buffer, and then to write it to the database or storage bucket. This batch processing allows you to cache the data efficiently, and saves you money by not constantly triggering the database read and write quotas.
Proper Use of IoT Rules
Consuming incoming data from device topics directly to a single service prevents systems from achieving full scalability. Sometimes, such an approach limits the availability of the system on events of failure and when there is too much data causing it to flood.
IoT Core Rules allows the system great flexibility in routing data to various AWS services in parallel. AWS services have different data flow properties and different use cases. These different services cannot be used as a single point of entry to the system, which is why Rules are important to properly identify and send data to the right places. Improper use of an AWS IoT service can lead to potential weak-points in your system that will hamper scalability and efficiency. It is vital to pay attention to data flow limitations and pipeline usages.
Conclusion
Knowing the data flow allows IoT architects to properly design systems using only the right AWS services. Efficiently designed IoT systems on AWS will use various AWS services for maximum synergy instead of piping all data through a single service or topic and saved to a single database. InCloud can design a great IoT system on AWS for you with our deep knowledge of all of the services and experience with various IoT data streams.