The promise of IoT solutions comes from their tremendous ability to harness data on a scale that has never before been possible. This data, wrangled by countless transmitters and sensors, offers us a wealth of insights about everything from the homes we live in to the products we buy to the health of our own bodies – all while IoT applications provide the power to act upon this data in real-time. 

Delivering these remarkable capabilities calls for a similarly capable database, one that can match IoT applications’ stringent requirements around performance, scalability, and availability. While these are the three core database concerns behind just about any application, I’d argue the IoT offers an even more mission-critical use case. Consider the magnitude of the data that IoT applications must process, along with the fact that many of these applications aren’t even functional if they cannot perform with near real-time fidelity – let alone become wholly unavailable – and it’s clear that IoT applications are introducing a new level of intensity when it comes to the demands placed on the database.

We know a thing or two about vetting and working with database solutions for IoT applications. Our project is the creation of an IoT Smart Kitchen Commerce solution –embedded within connected kitchen appliances – which links to grocery retailers and makes it easy to purchase needed items from within the customer’s own kitchen. Enabling this system’s simple-to-use front-end required tremendous back-end complexity. The database facilitating the solution needed to handle rich information on more than 1 million individual grocery products, which had all been collected by mapping retailers’ online product catalogs in granular detail. At its heart, this smart kitchen solution is all about the quality of the experience it offers users – whether it could truly make grocery shopping and managing kitchen restocking easier for customers – so its success absolutely depends on the database delivering real-time responsiveness and total reliability.

A major question in determining the specifics of the database we’d use for this project was the debate between an SQL or a NoSQL strategy. Each of these technologies has a lot to offer when applied to solutions for which they’re well-suited. Different IoT application use cases may benefit from utilizing SQL (relational) or NoSQL (non-relational) databases, and, to be both diplomatic and accurate, many projects might use both in tandem to great effect.

In our particular case, we discovered that NoSQL was the best fit for the task at hand, though the path to realizing it was a bit of a winding road. We actually began developing our solution by using MySQL as the database, managed by our internal team. Unfortunately, this led to a lot of difficulties. We found MySQL replication and other  administrative work to be disagreeable and full of thorny issues.

To alleviate what was becoming quite a pain point, we then decided to shift to a NoSQL database, recognizing that open source MongoDB would fit our solution much better, for reasons I’ll elaborate on. Also, having learned that managing the database internally wasn’t preferred or a great use of our resources, we enlisted mLab to be our MongoDB Database-as-a-Service provider. This accomplished two things: it made sure that the move to NoSQL MongoDB was handled seamlessly by experts, and provided our team the bandwidth to dive headlong into product development, which was really the best use of our team’s time.

NoSQL made sense in our case because the many data providers we work with use different data schemas. This understandably had created issues with MySQL, which requires defined schemas prior to accepting data. For enterprises in this position, our advice is to utilize the advantages of a NoSQL database in providing denormalized documents to quickly and easily accept any data in any format.

In addressing the key issue of scalability, a NoSQL approach has some effective advantages for dealing with the vast magnitude of IoT datasets while maintaining high performance. With NoSQL, for instance, it’s possible to utilize however many sharded servers there’s a need for, while each single server can maintain a limited and pre-determined size, making scaling easy to execute. It’s also worth saying that, because NoSQL shifts a good deal of logic to the application and away from the database, it’s easier for enterprises to recruit the talent they need. Great Java or C# coders outnumber great database programmers, and the expertise to optimize performance for complex queries is a rare and valued skill. While this remains true for any database, implementing a NoSQL strategy makes it that much less challenging to put a team in place with the right skillsets. It’s also the case that NoSQL is rapidly rising in popularity throughout the industry – with this rise, the tooling, frameworks, and knowhow required to best utilize NoSQL are becoming increasingly prevalent as well.

As we’ve discovered through our experience with MongoDB, the non-relational database has proven appropriate to meeting the huge data-handling demands of always-on IoT applications. Again, relational databases may offer a better fit for certain IoT applications, such as those working with less sizeable or dynamic datasets. But for our particular application – by handling vast and dynamic datasets regardless of their structure – our open source, NoSQL MongoDB approach provides the high-speed read and write performance, scalability, and high availability needed to deliver positive and effective real-time experiences for IoT consumers.

Phillip Zito at the highly resourceful blog Building Automation Monthly has consolidated multiple IoT Frameworks and Offerings into the IoT Database. You will see links to the Frameworks and Offerings below. He says over time that he will be working on providing summary articles on each Framework and Offering. He could use your help. If you have an offering/framework you would like added to this list feel free to add it in the comments. You can find the IoT Database here.