Three Major IoT Data-Related Challenges and How to Address Them

IoT has formed itself a crucial component for future-oriented solutions and holds a massive potential of economic value. McKinsey & Company estimates that by 2030, IoT (Internet of Things) will enable $5.5 trillion to $12.6 trillion in value worldwide, including the value captured by consumers and customers. For proof of its growing popularity and consumers’ dependency on it, you likely don't need to look any further than your own wrist. From fitness bands to connected vehicles, smart homes, and fleet-management solutions in manufacturing and retail, IoT already connects billions of devices worldwide, with many more to come.

As more IoT enabled devices come online, with increasingly sophisticated sensors, choosing the right underlying technology to make IoT solutions easier to implement and help companies seize new innovative opportunities is essential.

In this blog we will discuss how MongoDB has successfully addressed three major IoT data-related challenges across various industries, including Manufacturing, Retail, Telecommunications, and Healthcare. The challenges are the following:

  • Data Management

  • Real-Time Analytics

  • Supply Chain Optimization

FIgure 1: MongoDB Atlas for IoT

Let's dive right in!

Data management

Storing, transmitting, and processing the large amount of data that IoT devices produce is a significant challenge. Additionally, the data produced by IoT devices often comes in variable structures. This data must be carefully timestamped, indexed, and correlated with other data sources to make the context required for effective decision-making. This data volume and complexity combination makes it difficult to effectively and efficiently process data from IoT devices.

Bosch

Consider Bosch IoT Suite, a family of products and services in IoT device management, IoT data management, and IoT edge by Bosch Digital. These products and services hold over 250 international IoT projects and over 10 million connected devices.

Bosch implemented MongoDB to store, manage, and analyze data in real time. MongoDB’s ability to handle structured, semi-structured, and unstructured data, and efficient data modeling with JSON make it easy to map the information model of each device to its associated document in the database. In addition, dynamic schemas support agile development methodologies and make it simple to develop apps and software. Adding new devices, sensors, and assets is easy, which means the team can focus on creating better software.

ThingSpace

Another example is that of ThingSpace, Verizon’s market-leading IoT connectivity management platform, which provides the network access required to deliver various IoT products and services. Verizon works with companies that purchase network access from it to connect their devices, bundled together with their own solutions, which they sell to end users. ThingSpace’s customers each sell an IoT product that needs reliable connectivity to ensure the devices always work, which WiFi cannot offer.

Verizon’s monolithic RDBMS-based system would not be able to scale to handle both transactional and time-series workloads, so Verizon decided it needed a distributed database architecture going forward. MongoDB proved to be the only solution that scaled to meet Verizon’s requirements across different use cases and combinations of workload types. The immense processing needs resulting from the high number of devices and high velocity of messages coming in were only addressed by MongoDB’s highly available, scalable architecture. Native MongoDB Time Series allow for improved performance, through optimized storage with clustered indexes and optimized Time-Series query operators.

MongoDB's advanced capabilities, such as flexible data modeling, powerful indexing and Time Series provide an effective solution for managing the complex and diverse data generated by IoT devices.

Real-time analytics

Real-time data analytics, one of the most crucial parts of big data analytics today, brings value to businesses for making more data-driven real-time decisions. However, despite its importance, very few can respond to changes in data minute by minute or second by second. Many challenges arise when it comes to the implementation of real-time analytics for enterprises. Storing such a huge volume of data and analyzing it in real time is an entirely different story.

Thermo Fisher Cloud

Let’s consider the Thermo Fisher Cloud, one of the largest cloud platforms for the scientific community on AWS. MS Instrument Connect allows Thermo Fisher customers to see live experiment results from any mobile device or browser. Each experiment produced millions of "rows" of data, which led to suboptimal performance with existing databases. Internal developers needed a database that could easily handle a wide variety of fast-changing data. MongoDB's expressive query language and rich secondary indexes provided the flexibility to support both ad-hoc and predefined queries customers needed for their scientific experiments.

Anytime I can use a service like MongoDB Atlas, I’m going to take that so that we at Thermo Fisher can focus on what we’re good at, which is being the leader in serving science.

Joseph Fluckiger, Sr. Software Architect @Thermo Fisher

MongoDB Atlas scales seamlessly and is capable of ingesting enormous amounts of sensor and event data to support real-time analysis for catching any critical events or changes as they happen. That gives organizations new capabilities, including:

  • Capturing streaming or batch data of all types without excessive data mapping

  • Analyzing data easily and intuitively with a built-in aggregation framework

  • Delivering data insights rapidly and at scale with ease

With MongoDB organizations can optimize queries to quickly deliver results to improve operations and drive business growth.

Supply chain optimization

Items move through different locations in the supply chain, making it hard to maintain end-to-end visibility throughout their journey. The lack of control on any stage can dramatically harm the efficiency of planning, slow down the entire supply chain and ultimately result in lower return on investment. From optimizing warehouse space by sourcing raw materials as needed, to real-time supply chain insights, IoT-enabled supply chains can help significantly optimize these processes by eliminating blind spots and inefficiencies.

Longbow Advantage

Longbow Advantage delivers substantial business results by enabling clients to optimize their supply chains. Millions of shipments move through multiple warehouses every day, generating massive quantities of data throughout the day that must be analyzed for real-time visibility and reporting. Its flagship warehouse visibility platform, Rebus, combines real-time performance reporting with end-to-end warehouse visibility and intelligent labor management.

Longbow needed a database solution that could process quantities of that scale and deliver real-time warehouse visibility and reporting at the heart of Rebus, and it knew it could not rely on monolithic, time-consuming spreadsheets to do so. It became clear that MongoDB’s document database model was a good match and would allow Rebus to gather, store, and build visibility into disparate data in near real time.

Another key component of smart supply chain solutions is IoT-enabled mobile apps that provide real-time visibility and facilitate on-the-spot data-driven decisions. In such situations, the offline first paradigm becomes crucial, since staff need access to data in areas where connectivity is poor or nonexistent. Realm by MongoDB is a lightweight, object-oriented embedded database technology for resource constrained environments. It is an ideal solution for storing data on mobile devices. By utilizing MongoDB’s Realm SDKs, which wrap the Realm database, and Atlas Device Sync, which enables seamless data synchronization between MongoDB and Realm on your mobile phone with minimal developer efforts, businesses can rapidly develop mobile applications and drive innovation.

MongoDB provides a powerful solution for IoT-enabled supply chains that can optimize processes and eliminate inefficiencies, enabling organizations to make data-driven decisions and improve supply chain efficiency.

Conclusion

The IoT industry is rapidly evolving, and as the number of connected devices grows, so do the challenges faced by businesses leveraging these solutions.

Through a range of real-world use cases, we have seen how MongoDB has helped businesses deal with IoT data management, perform real-time analytics and optimize their supply chains, driving innovation in a variety of industries. With its unique features and capabilities, designed to manage the heavy lifting for you, MongoDB is well-positioned to continue playing a crucial role in the ongoing digital transformation of the IoT landscape.

Want to learn more or get started with MongoDB? Check out our IoT resources

← Previous

Bosch IoT and the Importance of Application-Driven Analytics

This post is also available in: 简体中文 For Kai Hackbarth and his team at Bosch, there exists the ability to bring real-time analytics into their applications that can handle lots and lots of data. “Without data, it’s just trial and error.” So says Kai Hackbarth . He’s a senior technology evangelist at Bosch Global Software with over 22 years of industry experience in the Internet of Things. “Be it automotive, be it industrial, be it smart-home, we’ve done it all,” Hackbarth said. Except for tires. That’s his latest focus and technology challenge. “Sounds maybe a bit simple,” Hackbarth said, “but if you think about it more deeply, [it’s really complex].” Because, as it turns out, tires can collect a lot of different pieces of data that can tell a system a lot about what’s going on with a lot of different things related to the car at any given moment. “Pressure, temperature, accelerometer,” Hackbarth said. “And then you also have other data from the car that’s critical for safety and sustainability.” But to be of any value, that data needs to be analyzed as close to the source as possible and in real time. Why? “It’s safety-critical,” Hackbarth said. “If you send all the raw data to the cloud this consumes a lot of costs.” Chief among those costs: Time. In order to react to an issue, that data cannot be historical. Because historical data about a tire that’s losing pressure or hydroplaning isn’t helpful to the applications inside the car that need to respond to these developments when they’re happening And thankfully for Hackbarth and his team at Bosch, there exists the ability to bring real-time analytics into their applications that can handle lots and lots of data. Smarter applications start with built-in analytics Traditionally, applications and analytics have existed as two separate classes of workloads with different requirements, for example: read and write access patterns, as well as concurrency and latency. As a result, businesses have usually deployed purpose-built data stores — including both databases for applications and data warehouses for analytics — and piping or duplicating the data between them. And that’s been fine when analytics don’t need to affect how an application responds in real time. But most customers expect applications to take intelligent actions in the moment, rather than after the fact. The same principle applies to Bosch’s tire project. The applications inside the car that can autonomously brake when approaching another vehicle too fast or slow down if the tire senses that it’s hydroplaning, need to also analyze all the data from all the sensors in real time. This process of building real-time analytics into applications is known as “ application-driven analytics .” And it’s how applications get smarter. Be they e-commerce apps on your phone, safety apps in your car or those that monitor rocket launches . The question for many development teams, though, is how do you build this capability into your applications easily? For a long time, that’s been a difficult question to answer. A platform for building real-time analytics into your apps “From my history [of doing this for] 22 years,” Hackbarth says, “we never had the capabilities to do this before.” Previously, teams everywhere — not just at Bosch — used to have to do a lot of custom engineering work to do real-time analytics close to the source, including: Stitching together multiple databases to handle different data structures (documents, tables, time series measurements, key values, graph), each accessed with its own unique query API. Building ETL data pipelines to transform data into required analytics formats and tier it from the live database to lower-cost object storage . Spinning up a federated query engine to work across each data tier, again using its own unique query API. Integrating serverless functions to react to real-time data changes. Standing up their own API layers to expose data to consuming applications. All of which result in a multitude of operational and security models to deal with, a ton of data integration work and lots of data duplication. But that’s no longer the case. There are now data platforms that bring together operational and analytical workloads into one, and that, in turn, allow you to bring live, operational data and real-time analysis together. MongoDB Atlas — the platform with which I am most familiar (since I work at MongoDB) — allows developers to do this by providing an integrated set of data and application services that fit their workflows. Developers can land data of any structure, index, query and analyze it in any way they want, and then archive it. And they can do all of this with a unified API and without having to build their own data pipelines or duplicate data. This is the platform on which the Bosch team continues to build its solutions. “Without data,” Hackbarth says, “it’s just trial and error.” But now with data and with a single platform to build real-time analytics into their applications, it’s something concrete, something responsive and something actionable. It’s something smart. Especially in the most critical of use cases that Bosch is working on. Like tires. If you’d like to learn more about how to build application-driven analytics into your applications, check out our three-part livestream demo showing how to build application-driven analytical applications in MongoDB Atlas . During this three-part series, we build real-time analytical capabilities for a simulated system for managing rocket launches. Part One covers the basics: Building complex analytical queries using MongoDB’s aggregation framework and building visualizations using charts. Part Two highlights some additional Atlas capabilities that are often invaluable when building app-driven analytics: Atlas search, triggers and embedding Charts generated visualizations into an application UI. Part Three focuses on how to use Atlas Data Federation, Atlas Data Lake and Atlas SQL Interface to perform analytics using large sets of historical data and federated queries across multiple data sources.

April 19, 2023
Next →

The Journey of MongoDB with COVESA in the Connected Vehicle Landscape

There’s a popular saying: “If you want to go fast, go alone; if you want to go far, go together.” I would argue The Connected Vehicle Systems Alliance (COVESA) in partnership with their extensive member network, turns this saying on its head. They have found a way to go fast, together and also go far, together. COVESA is an industry alliance focused on enabling the widespread adoption of connected vehicle systems. This group aims to accelerate the development of these technologies through collaboration and standardization. It's made up of various stakeholders in the automotive and technology sectors, including car manufacturers, suppliers, and tech companies. COVESA’s collaborative approach allows members to accelerate progress. Shared solutions eliminate the need for individual members to reinvent the wheel. This frees up their resources to tackle new challenges, as the community collectively builds, tests, and refines foundational components. As vehicles become more connected, the data they generate explodes in volume, variety, and velocity. Cars are no longer just a mode of transportation, but a platform for advanced technology and data-driven services. This is where MongoDB steps in. MongoDB.local NYC Join us in person on May 2, 2024 for our keynote address, announcements, and technical sessions to help you build and deploy mission-critical applications at scale. Use Code Web50 for 50% off your ticket! Learn More MongoDB and COVESA As the database trusted for mission-critical systems by enterprises such as Cathay Pacific , Volvo Connect , or Cox Automotive ; MongoDB has gained expertise in automotive, along with many other industries, building cross-industry knowledge in handling large-scale, diverse data sets. This in turn enables us to contribute significantly to vehicle applications and provide a unique view, especially in the data architecture discussions within COVESA. MongoDB solutions support these kinds of innovations, enabling automotive companies to leverage data for advanced features. One of the main features we provide is Atlas Device SDKs : a low-footprint, embedded database directly living on ECUs. It can synchronize data automatically with the cloud using Atlas Device Sync , our data transfer protocol that compresses the data handles conflict resolution, and only syncs delta changes, making it extremely efficient in terms of operations and maintenance. VSS: The backbone of connected vehicle data An important area of COVESA’s work is the Vehicle Signal Specification (VSS). VSS is a standardized framework used to describe data of a vehicle, such as speed, location, and diagnostic information. This standardization is essential for interoperability between different systems and components within a vehicle, as well as for external communication with other vehicles and infrastructure. VSS has been gaining more and more adoption, and it’s backed by ongoing contributions from BMW, Volvo Cars, Jaguar LR, Robert Bosch and Geotab, among others. MongoDB’s BSON and our Object-oriented Device SDKs uniquely position us to contribute to VSS implementation. The VSS data structured maps 1 to 1 to documents in MongoDB and objects in Atlas Device SDKs , which simplifies development, and speeds up applications by completely skipping any Relational Mapper layer. For every read or write, there is no need to transform the data between relational and VSS. Our insights into data structuring, querying, and management can help optimize the way data is stored and accessed in connected vehicles, making it more efficient and robust. Where MongoDB contributes MongoDB, within COVESA, finds its most meaningful contributions in areas where data complexities and community collaboration intersect. First, we can share insights into managing vast and varied data emerging from connected vehicles generating data on everything from engine performance to driver behavior. Second, we have an important role in supporting the standardization efforts, crucial for ensuring different systems within vehicles can communicate seamlessly. Our inputs can help ensure these standards are robust and practical, considering the real-world scenarios of data usage in vehicles. Some of our contributions include an Over the Air update architectural review presented at Troy COVESA’s AMM in October 2023; sharing insights about the Data Middleware PoC with BMW; and weekly contributions at the Data Expert Group. You can find some of our contributions on COVESA’s Wiki page . In essence, MongoDB's role in COVESA is about providing a unique perspective from the database management point of view, offering our understanding from different industries and use cases to support the developments towards more connected and intelligent vehicles. MongoDB, COVESA, and AWS together at CES2024 MongoDB’s most recent collaboration with COVESA was at the Consumer Electronics Show CES 2024 during which MongoDB’s Connected Vehicle solution was showcased. This solution leverages Atlas Device SDKs, such as the SDK for C++ , which enables local data storage, in-vehicle data synchronization, and also uni and bi-directional data transfer with the cloud. Below is a schematic illustrating the integration of MongoDB within the software-defined vehicle: Schema 1: End to end integration for the connected vehicle At CES 2024, MongoDB also teamed up with AWS for a compelling presentation, " AI-powered Connected Vehicles with MongoDB and AWS " led by Dr. Humza Akhtar and Mohan Yellapantula, Head of Automotive Solutions & Go To Market at AWS. The session delved into the intricacies of building connected vehicle user experiences using MongoDB Atlas. It showcased the combined strengths of MongoDB's expertise and AWS's generative AI tools, emphasizing how Atlas Vector Search unlocks the full lifecycle value of connected vehicle data. During the event, MongoDB also engaged in a conversation with The Six Five, exploring various aspects of mobility, self-driving vehicles (SDVs), and the MongoDB and AWS partnership. This discussion extended to merging IT and OT, GenAI, Atlas Edger Server, and Atlas Device SDK. Going forward At the end of the road, it’s all about enhancing the end-user experience and providing unique value propositions. Defect diagnosis based on the acoustics of the engine, improved crash assistance with mobile and vehicle telemetry data, just-in-time food ordering while on the road, in-vehicle payments, and much, much more. What all these experiences have in common is the combination of interconnected data from different systems. At MongoDB, we are laser-focused on empowering OEMs to create, transform, and disrupt the automotive industry by unleashing the power of software and data. We enable this by: Partnering with alliances such as COVESA to build a strong ecosystem or collaboration. Having one single API for In-vehicle Data Storage, Edge to Cloud Synchronization, Time Series storage, and more, improves the developer experience. Focusing on having a robust, scalable, and secure suite of services trusted by tens of thousands of customers in more than 100 countries. Together with COVESA’s vision for connected vehicles, we’re driving a future where this industry is safer, more efficient, and seamlessly integrated into the digital world. The journey is just beginning. To learn more about MongoDB-connected mobility solutions, visit the MongoDB for Manufacturing & Motion webpage . Achieving fast, reliable and compressed data exchange is one of the pillars of Software Defined Vehicles, learn how MongoDB Atlas and Edge Server can help in this short demo .

April 15, 2024