Dr. Humza Akhtar

6 results

Building an Industrial Unified Namespace Architecture with MongoDB and Arcstone

Thank you to Karolina Ruiz Rogelj for her contributions to this post. The fourth industrial revolution, also known as Industry 4.0 is rapidly transforming the manufacturing industry. Leveraging I4.0 reference architectures and Industrial IoT technologies, factories generate more data than ever. Market analyst reports tell us that the global number of Industrial IoT connections will increase to 36.8 billion in 2025. As factories become more connected and data-driven, it is essential to have a unified and standardized approach for manufacturing data management. In this article, we explain how MongoDB helps create a Industrial Unified Namespace (IUN) architecture that can act as a contextualized repository for data and information for all manufacturing assets. Manufacturing companies have been leveraging the International Society of Automation’s standard 95 (ISA-95) to develop automated interfaces between industrial control systems and enterprise systems. ISA-95 provides a hierarchical model for interfacing and integration also known as the automation pyramid. Figure 1 shows the five levels of the automation pyramid. Figure 1: ISA-95 Automation Pyramid. ISA-95 was introduced in 2000 to improve communication and data exchange between different levels of the manufacturing industry. With the advent of Industrial IoT (IIoT), the limitations of the ISA-95 model have become increasingly apparent. Lack of Interoperability: The model was developed for a more traditional, hierarchical approach to manufacturing, where there is a clear separation between operational technology (OT) and information technology (IT). In contrast, IIoT tries to blur the lines between OT and IT, with a greater emphasis on data interoperability and real-time analytics. Limited Flexibility and Agility: The rigid and hierarchical structure imposed by the automation pyramid goes against Industry 4.0 concepts of flexibility and agility. The data captured by sensors must go through the SCADA and MES layers to reach the top level. This makes it difficult for manufacturers to adapt to changing production requirements and integrate IIoT technology into their existing systems. Limited Scalability: The ISA-95 model was designed for a traditional manufacturing environment with a limited number of production lines and machines. However, with the growth of Industry 4.0, the number of connected devices and the amount of data generated has increased dramatically. The automated pyramid does not easily scale to handle this increased volume of data and devices, leading to potential bottlenecks and inefficiencies in the manufacturing process. For example, if a new machine is added to the production line, ISA-95 requires significant changes to the factory IT and OT architecture, which can be time-consuming and costly. Industrial unified namespace (IUN) architecture with MongoDB In order to overcome these challenges, we propose that manufacturers adopt an Industrial Unified Namespace (IUN) architecture leveraging MongoDB technology. Such an architecture will provide a single view of all manufacturing processes and equipment and will enable data interoperability between different layers of the ISA-95 automation pyramid. Figure 2 shows a conceptual diagram of the IUN architecture. Figure 2: Event driven industrial unified namespace IUN follows an event-driven architecture topology where different manufacturing applications publish events in real-time (publishers) to the central MongoDB Atlas database. Application services subscribe asynchronously to the event types or topics of interest and consume them at their own speed (consumers). This results in a decoupled ecosystem allowing applications and services to act interchangeably to provide and consume data when and where needed in real-time. It is understood that many applications and services may produce and consume data at the same time. MongoDB Atlas database plays a central role in the IUN architecture. The events can flow in through MongoDB Kafka Connector or Atlas Device Sync and MongoDB Atlas can aggregate, persist and serve them to consuming manufacturing applications. The core MongoDB Atlas database in this scenario provides a central repository for multiple independent event streams and the developer data platform helps to drive operational and analytical apps providing a complete end-to-end view of the production process. Data modeling for industrial unified namespace The document model is the most natural way to work with data stored in the database. It is simple for any developer to learn how to code against MongoDB, and as a result, industry surveys show it is wildly popular amongst developers. MongoDB provides flexible data modeling options to create a central repository for all factory production data. Asset-centric data model: Focusses on the assets, for example machines, equipment, tools in the manufacturing process. This data model is useful for tracking the performance, maintenance, and utilization of assets. Process-centric data model: Focuses on the day to day production processes. Such a data model is useful in optimizing the process flow and reducing bottlenecks. Product-centric data model: Focuses on the products produced in the manufacturing process. This data model is useful for tracking the production and quality of individual products. It is possible for a factory to have all three models at the same time. In fact, it is common for factories to use multiple data models and integrate them as needed to gain a complete view of their operations. For example, a factory may use an asset-centric model to track its equipment, and a product-centric model to track its finished goods, while also using a process-centric model to optimize its manufacturing processes. Let us take an example of a bicycle factory and look at example asset, process and product-centric data models. At a minimum, the following collections (Figure 3) will need to be created in the database. Figure 3: MongoDB collections for different IUN data models Each collection will have data coming from different sources such as Manufacturing Execution System (MES), IIoT Platform, and Enterprise Resource Planning (ERP) systems. An example document from the production equipment collection is shown in Figure 4. As it can be seen, the data comes from various sources and the MongoDB document model makes it very easy to combine this data together in one document generating a digital twin prototype of the machine. Figure: A sample document from the Production Equipment collection Architecture for industrial unified namespace Let us take our bicycle factory and create a solution architecture for the Industrial Unified Namespace. First, let us list down all the event producers and consumers. All these systems both consume and publish events: IoT Gateways / Edge Server Supervisory Control and Data Acquisition (SCADA) / Shop Floor Connectivity Platform (SCP) Manufacturing Execution System (MES) Enterprise Resource Planning (ERP) Arcstone toolsets for smart manufacturing Arcstone is a Singapore/US-based Industry 4.0 solutions company providing modular-based, next-generation MES alongside hardware integration and process orchestration toolsets. Arcstone delivers success to companies from diverse industries, including Global Fortune 500 companies, manufacturing companies, emerging facility management firms, and SMEs, globally. Arcstone arc.ops MES contains 15+ modules for full operational management that can be custom tailored to specific requirements, and is built to be end-user configurable for easy intuitive use. Arcstone understands that extracting data from legacy equipment is a challenging task. Therefore, they have created a low-code solution named arc.quire to handle the collection of raw data and streaming into a database for storage. arc.quire is used in tandem with a process orchestration tool called arc.flow to establish connectivity between arc.quire and the database, for example, MongoDB EA. Depending on the connectivity interface exposed by the production equipment, SCADA or SCP software can connect to the equipment and push the raw events and alerts to the arc.quire running in the edge server. MongoDB’s Enterprise Operator for Kubernetes , gives the flexibility to run MongoDB as a container in resource-constrained environments such as our IoT edge server. Figure 5 shows how the edge server can be connected with the SCADA and IoT gateways on the production shop floor. Figure 5: Edge Server with MongoDB and Arcstone toolsets The edge server performs the following functions: Aggregation of IIoT events and alerts via arc.quire Real-time analytics such as machine fault detection, process optimization, and process control via the MongoDB aggregation framework Transmitting control instructions back to the equipment via arc.quire Raw data and analytical results storage in MongoDB Edge servers act as one of the event producers for IUN. Using the MongoDB Kafka connector, events can be transmitted from the edge server to a centralized data repository in MongoDB Atlas. Figure 6: MongoDB can serve as both a Sink and a Source for Apache Kafka Bringing it all together Figure 7 shows the complete technical architecture of the Industrial Unified Namespace with MongoDB Atlas Developer Data Platform and Arcstone. Figure 7: In this architecture, arc.ops MES, ERP, and edge server publish data to the message stream in Apache Kafka where the event queue makes the data available for MongoDB Atlas to consume via Kafka connectors [1 and 2]. Depending on the factory requirements around batch processing and scalability, Kafka can be replaced by a MQTT broker. There are multiple community backed and commercial libraries to push MQTT data into MongoDB. The centralized database aggregates and persists events, enriches event streams with data from all sources, including historical data, and provides a central repository for multiple event streams. This enables applications and users to benefit from all data across all microservices and provides a unified view of the state across the factory. Atlas also leverages Atlas Charts for events visualization as well as Atlas Search for full-text search of events [3 and 4]. MongoDB’s Atlas Triggers provide a serverless way of consuming change stream events [5]. With Triggers, the manufacturer doesn't have to set up their own application server to run your change data capture process. Change streams flow change data to Atlas Triggers to create responsive, event-driven pipelines. Finally, Atlas Device Sync and Realm SDK can be leveraged to push real-time notifications and alerts to shop floor applications for users to consume. Use cases Predictive maintenance IUN can be deployed as the foundation for predictive maintenance applications. Edge server streams time-series event data from the production equipment into MongoDB to drive machine-learning models that will detect equipment health and performance degradation trends. The data is enriched using data streams about production jobs from MES. The factory can either repair equipment or swap it out for replacement parts before shutting down production lines. Atlas Device Sync can alert engineers on the shop floor to potential equipment failures, and help the company optimize the equipment maintenance strategy. Operational data layer The IUN architecture can be used to create a manufacturing Operational Data Layer (ODL). An ODL strives to centrally integrate and organize all siloed manufacturing IT/OT data and makes it accessible to stakeholders across the factory floor. This ODL will combine data from both OT and IT sources into a single MongoDB Atlas database where Atlas Search and Charts can be used to analyze this data and drive actions on the shop floor. IUN captures any changes in source systems and streams them into MongoDB to keep the ODL fresh, and helps to update the source systems in real-time. Conclusion In conclusion, the ISA95 Automation Pyramid presents significant challenges for the manufacturing industry, including a lack of flexibility, limited scalability, and difficulty integrating new technologies. By adopting an Industrial Unified Namespace architecture with Arcstone and MongoDB, manufacturers can overcome these challenges and achieve real-time visibility and control over their operations, leading to increased efficiency and improved business outcomes. To learn more about MongoDB’s role in the manufacturing industry, please visit our Manufacturing and Industrial IoT page.

April 27, 2023

Connected Devices - How GE HealthCare Uses MongoDB to Manage IoT Device Lifecycle

GE HealthCare, a global leader in medical technology, has turned to MongoDB to manage the lifecycle of its IoT devices, from deployment (Beginning of Life or BoL) to retirement (End of Life or EoL). At GE HealthCare, MongoDB Atlas is used to persist device and customer data. These related data layers are utilized by the organization to develop customer experience strategies by providing greater efficiency, improving patient outcomes, and increasing access to care. The MongoDB document model easily combines data from diverse source systems while preserving its full fidelity. This flexibility allows seamless onboarding of new customers and related data sources without requiring time consuming schema modifications. According to Emir Biser, Senior Data Architect at GE HealthCare, MongoDB Atlas is very appealing to the team because of its effective management, built-in monitoring and backup, global vertical and horizontal scalability, built-in security, and multi-cloud support. MongoDB Atlas is a gamechanger. This technology stack is helping us streamline commercialization and bring market-ready solutions to deliver advanced healthcare. Some of the recent tests resulted in an *83% decrease in retrieval time for critical data elements. When all these features are put together, the tech stack is designed to help healthcare providers enhance productivity by reducing the complexity and time required to manage databases, enabling faster deployment of IoT devices. Enhancing the IoT life cycle with MongoDB GE HealthCare’s tech-stack is designed to accelerate the integration of healthcare applications by connecting IoT devices together with additional data sources into an aggregated clinical data layer. As the IoT device connections are established, multiple services are applied on the platform to support analytic and clinical applications. Beginning of life - Device provisioning and configuration As the device is being manufactured, the device parameters such as MAC and serial number are stored in MongoDB as a device digital representation. When the device is turned on, the GEHC team gets information about the device usage and the customer information. This information is used to validate the device. MongoDB is playing a crucial role in device provisioning by persisting the configuration information and making sure that the device is set up with the right configuration parameters. MongoDB change streams are used at this stage to make sure that the device gets the right parameters at the BoL stage. Middle of life - Device usage and maintenance Once the device comes online, it transmits both clinical and non-clinical information. The team at GE HealthCare uses MongoDB Atlas to help ensure clear separation between clinical and non-clinical as permissions, sensitivity, and access differs. Additionally, to understand how the device is being used compared to its standard configuration parameters. MongoDB’s real-time analytics capabilities help track key device performance metrics, such as battery life and identify trends and patterns in device usage. This enables the team to proactively address device issues, improve overall device performance and reliability for customers. GEHC is able to share these insights with customers to help optimize use of devices within their enterprise. MongoDB Atlas Search is used to retrieve information about status of connected devices and usage patterns. Search Compound Geo JSON queries are used to look at products in a certain geographic region. Horizontal scalability with automatic sharding across clusters ensures Edison applications can continue to be cost effective while delivering real-time results. MongoDB’s security features, including authorization, authentication and encryption, work with GEHC processes to enable teams working to protect device data from unauthorized access. End of life - Device decommissioning and archiving When the IoT device reaches the end of its lifecycle, GE HealthCare needs to decommission it and ensure that any data associated with the device is securely archived. By using MongoDB’s TTL (time-to-live) collections feature, the team automates the process of data deletion, reducing the data footprint. In addition, Atlas Online Archive helps to ensure that the data is always backed up and securely archived, reducing the risk of data loss and corruption. The authentication and authorization mechanisms help to ensure that decommissioned devices data can only be accessed by authorized personnel. The future of GE HealthCare According to Emir, the teams using MongoDB Atlas are excited about the benefits it brings, and they are looking forward to exciting new developments in Atlas platform. We are helping teams achieve business goals across Imaging, Ultrasound Digital Solutions, and Patient Care Solutions. Our current strategy focuses on building solid pipelines to further help our medical device engineering teams deliver interoperability resulting in better care for our customers. More on managing massive IoT devices Internet of Things (IoT) is transforming the healthcare industry by providing real-time, actionable insights that improve patient outcomes and drive operational efficiencies. According to market analyst reports , the global IoT healthcare market is projected to reach around USD 446.52 billion by 2028 while exhibiting a CAGR of 25.9% between 2021 and 2028. In hospitals, IoT-enabled medical devices help improve patient safety and clinical experience by transmitting real-time monitoring and alerts in the event of device malfunctions or irregularities. The life of an IoT device can be divided into three main stages: Beginning of Life (BoL), Middle of Life (MoL) and End of Life (EoL). During the BoL stage, the key activities are deployment design and provisioning. In this stage the device may be pre loaded with default credentials and configuration files. Once the device is installed and comes online, the focus in the MoL is to maintain its basic functional purpose as well as regularly updating firmware for reliability and security purposes. Over time, as new versions of devices are manufactured, the deployed devices need to be decommissioned by revoking the device certificate, archiving device data and disabling the model of device in the cloud as part of the EoL stage. Figure 1: Three stages of IoT device lifecycle management In each of the stages, the device has to be maintained to stay reliable, efficient, persistent and secure. Setting up telemetry from device to cloud/back end is just the tip of the iceberg. As the number of IoT devices deployed in healthcare continues to grow, so does the challenge of managing them efficiently. The large amount of data generated creates scalability challenges for IoT device management systems, which need to be able to handle large amounts of data and support the increased traffic. Different communication protocols make it challenging to integrate these devices into a unified system. Maintaining standard communication protocols and interoperability is critical to ensure seamless communication between devices and cloud backend. Finally, with the increasing number of cyber-attacks targeting IoT devices, it is critical to have robust security measures in place to protect against threats. To learn more about GEHC digital offerings please visit https://apps.gehealthcare.com/ Test performed internally by GE HealthCare on company datasets and may not be replicable. To learn more about MongoDB’s role in the healthcare and manufacturing industry, please visit our Manufacturing and Industrial IoT and Healthcare pages.

April 25, 2023

How MongoDB, A*STAR, and Industry Partners are Collaborating on Singapore’s Supply Chain 4.0 Initiative

Greater uncertainty in global trade flows and black swan events, such as COVID-19, have challenged the linear supply chain business model. Digital technologies are being recognized as a key enabler for resilient and responsive supply chains. Supply Chain 4.0 is the reorganization of supply chain – plan, source, make, deliver, return and enable from a linear business model to an integrated one using concepts of Industry 4.0 (I4.0). In this article, we’ll explore how MongoDB, together with our industry partners in Singapore, help businesses integrate technological innovations into their operations to deal with diverse challenges posed by growing supply chains. Supply chain trends and challenges In today’s world of uncertainties and disruptions, manufacturing supply chains are becoming increasingly complicated and opaque. This is happening alongside organic supply chain evolution involving digitalisation, unified ecommerce and sustainability awareness. Disruptions are costly to deal with, often requiring manufacturers to expend large amounts of (and sometimes evitable) resources correcting them. Many companies were unprepared for the shockwaves from COVID-19 global crisis and realized that they should not take the supply chain for granted and they must invest in digitizing their supply chain operations. In recent years, the investment in digital technologies for supply chain planning and execution has increased considerably. The emergence of Cloud Computing and the Industrial Internet of Things (IIoT) has promoted new opportunities for the supply chain and logistics domain. For example, real-time, cloud-based logistics and transport management systems have made logistics services more responsive and efficient, especially for small and medium-sized companies. However, supply chain management is much more complex than just logistics tracking. A reliable Supply Chain 4.0 platform should have some of these capabilities: End to End Visibility: Aggregates data from various systems supporting supply chain planning and execution processes (e.g ERP, MES, WMS, TMS) and provides a single view for monitoring supply chain performance in real-time. Decision Making Support: Contains tools and algorithms to support decision making for operations such as production scheduling, inventory assignment and order fulfillment optimization etc. Disruption Prediction and Management: To be able to predict anomalies and respond in time upon major disruptions events by orchestrating tools for network simulation, production re-planning etc. There are certain challenges associated with building a Supply Chain 4.0 platform that can enable above mentioned capabilities. Data Collection and Privacy Challenges: Brand owners need the ability to track products, raw materials and goods across the supply chain to get a clear picture of inventory and supply chain overall health. They can use this data to predict and manage supply chain disruptions and risks. Building this ability is a daunting task as it requires sharing data between supply chain tiers while navigating the data privacy and security risks to get real-time global visibility across all supply chain nodes. A federated infrastructure might be the answer where the private raw data is kept locally and transformed data is synced with the cloud. Data Modelling and Compatibility Challenges: A supply chain 4.0 platform must cater for the integration of the huge number of devices and services across the entire supply chain. These services and devices will transmit varied data in large volumes. This poses a data modeling challenge where a heterogenous data store is required to store this large amount of structured and unstructured data together. Real-time Analytics Requirements: Supply chain real time use cases such as delivery dispatching, production scheduling, inventory management and logistics tracking requires tools and APIs that help companies build more sophisticated queries against live data of any shape and structure in addition to mechanisms to separate operational from analytical processing so the application doesn’t slow down, along with the ability to land insights close to users. With these challenges in mind, the A* STAR Advanced Remanufacturing and Technology Centre (ARTC) initiated a Supply Chain 4.0 Program with other partners, to develop digital and automation solutions to meet businesses’ demands for technologies to make supply chains more agile, resilient, and secure. A*STAR also opened a Supply Chain Control Tower, to testbed these solutions with partners. Supported by research partners, the initiative has attracted over 50 companies from across five sectors (aerospace, fast-moving consumer goods (FMCG), pharmaceuticals, precision engineering, and semiconductors), including multinational companies and local small and medium enterprises (SMEs). Together with other Supply Chain 4.0 partners, MongoDB is supporting ARTC in developing an easy-to-use database platform, ORCA, that can enable data sharing and processing across and within enterprises. There are two main components in ORCA: ORCA hub and ORCA edge. ORCA hub takes care of cross-enterprise information sharing and is built on federated database architecture in which a collection of independent database systems are united into a loosely coupled federation in order to share and exchange information. The approach would be a Hybrid between Cloud and Local resources. The cloud will only keep the metadata, models, references while keeping actual data locally to the organization eliminating massive data migration and data privacy concerns between member organizations. ORCA edge, on the other hand, takes care of enterprise information aggregation and enables integrating legacy SCM systems (ERP, TMS, WMS) via a novel data exchange middleware. It provides a seamless and synchronized communication environment for different simulation platforms and risk management. MongoDB Atlas has been leveraged among other technologies to develop this easy-to-use data platform. The document model makes it possible for the developers to model heterogeneous data coming in from multiple sources in the supply chain. Realm database acts as the persistence layer in ORCA edge and filtered collections are synced with ORCA hub database via Atlas Device Sync. GraphQL API is leveraged for read queries by nodes in the supply chain. Figure 1: ORCA data fabric - Overall architecture The ORCA platform can collate data from multiple sources in the supply chain, as well as enable quick information sync and search. The technology developed by the Supply Chain 4.0 Program could help companies mitigate supply chain disruptions. Visit our Manufacturing hub to learn more about innovation in the manufacturing space.

April 4, 2023

MongoDB Atlas as the Data Hub for Smart Manufacturing with Microsoft Azure

All the source code used in this project, along with a detailed deployment guide, is available on our public Github page . Manufacturing companies are emerging from the pandemic with a renewed focus on digital transformation and smart factories investment. COVID-19 has heightened the need for Industrial IoT technology and innovation as consumers have moved towards online channels, forcing manufacturers to compete in a digitalized business environment. The manufacturing ecosystem can be viewed as a multi-dimensional grouping of systems designed to support the various business units in manufacturing organizations such as operations, engineering, maintenance, and learning & development functions. Process and equipment data is generated on the shop floor from machines and systems such as SCADA and then stored in a process historian or an operational database. The data originating from shop floor devices are generally structured time series data acquired through regular polling and sampling. Historians provide fast insertion rates of time series data, with capacities that reach up to tens of thousands of PLC tags processed per second. They rely on efficient data compression engines which can either be lossy or lossless. Traditional RDBMS storage comes packaged with the manufacturing software applications such as a Manufacturing Execution System (MES). Relational databases are traditionally common in manufacturing systems and thus the choice of database systems for these manufacturing applications are typically driven by historical preferences. Manufacturing companies have long relied on using several databases and data warehouses to accommodate various transactional and analytical workloads. The strategy of separating operational and analytical systems has worked well so far and has caused least interference with the operational process. However this strategy will not fare well in the near future for two reasons: Manufacturers are generating high volume, variety and veracity data using advanced IIoT platforms to create a more connected product ecosystem. The growth of IIoT data has been rapid and in fact, McKinsey and Company estimates that companies will spend over $175B in IIoT and edge computing hardware by 2025. A traditional manufacturing systems setup necessitates the deployment and maintenance of several technologies including graph databases (for asset digital models and relationships) and time series databases (for time series sensor data) and leads to IT sprawl across the organization. A complex infrastructure causes latency and delays in data access which leads to non-realization of real time insights for improving manufacturing operations. To establish an infrastructure that can enable real time analytics, companies need real time access to data and information to make the right decision in time. Analytics can no longer be a separate process, it needs to be brought into the application. The applications have to be supplied with notifications and alerts instantly. This is where application-driven analytics platforms such as MongoDB Atlas come into picture. We understand that to build smarter and faster applications, we can no longer rely on maintaining separate systems for different transactional and analytical workloads. Moving data between disparate systems takes time and energy and results in longer time to market and slower speed of innovation. Many of our customers start out using MongoDB as an operational database for both new cloud-native services as well as modernized legacy apps. More and more of these clients are now improving customer experience and speeding business insight by adopting application-driven analytics within the MongoDB Atlas platform. They use MongoDB to support use cases in real-time analytics, customer 360, internet of Things (IoT) and mobile applications across all industry sectors. As mentioned before, Manufacturing ecosystem employs a lot of databases just to run production operations. Once IIoT solutions are added to the mix, each solution (shown in yellow in Figure 1) may come with its own database (Time Series, relational, graph etc.) and the number of databases will increase dramatically. With MongoDB Atlas, this IT sprawl can be reduced as multiple use cases can be enabled using MongoDB Atlas (Figure 2). The versatility of the document model to structure data any way the application needs, coupled with an expressive API and indexing that allows you to query data any way you want is a powerful value proposition. The benefits of MongoDB Atlas are amplified by the platform’s versatility to address almost any workload. Atlas combines transactional processing, application-driven analytics, relevance-based search, and mobile edge computing with cloud sync. These capabilities can be applied to almost every type of modern applications being built for the digital economy by developers. Figure 1: IT sprawl with IIoT and analytics solutions deployment in Manufacturing Figure 2: MongoDB Atlas simplifying road to Smart Manufacturing MongoDB and Hyperscalers leading the way for smart manufacturing Manufacturers who are actively investing in digital transformation and IIoT are experiencing an exponential growth in data. All this data offers opportunities for new business models and digital customer experiences. To drive the right outcomes from all this data, manufacturers are setting up scalable infrastructures using Hyperscalers such as Azure, AWS and GCP. These hyperscalers offer a suite of components for efficient, scalable implementation of IIoT platforms. Companies are leveraging these accelerators to quickly build solutions, which help access, organize, and analyze previously untapped data from sensors, devices, and applications. In this article, we are focused on how MongoDB integrates with Microsoft Azure IoT modules and acts as the digital data hub for smart manufacturing use cases. MongoDB and Microsoft have been partners since 2019, but last year it was expanded, enabling developers to build data intensive applications within the Azure marketplace and Azure portal. This enables an enhanced developer experience and allows burn down of their Microsoft Azure Consumption Commitment. The alliance got further boost when Microsoft included MongoDB as a partner in its newly launched Microsoft Intelligent Data Platform Ecosystem . MongoDB Atlas can be deployed in 35 regions in Azure and has seamless integration with most of the Azure Developer services (Azure functions, App services, ADS), Analytics services (Azure Synapse), Data Governance (Microsoft Purview), ETL (ADF) and cross cutting services (AD, KMS, AKS etc.) powering building of innovative solutions. Example scenario: Equipment failure prediction Imagine a manufacturing facility that has sensors installed in their Computer Numerical Control (CNC) machines measuring parameters such as temperature, torque, rotational speed and tool wear. A sensor gateway converts analog sensor data to digital values and pushes it to Azure IoT Edge which acts as a gateway between factory and the Cloud. This data is transmitted to Azure IoT Hub where the IoT Edge is registered as an end device. Once we have the data in the IoT Hub, Azure Stream Analytics can be utilized to filter the data so that only relevant information flows into the MongoDB Atlas Cluster. The connection between Stream Analytics and MongoDB is done via an Azure Function. This filtered sensor data inside MongoDB is used for following purposes: To provide data for machine learning model that will predict the root cause of machine failure based on sensor data. To act as a data store for prediction results that can be utilized by business intelligence tools such as PowerBI using Atlas SQL Interface. To store the trained machine learning model checkpoint in binary encoded format inside a collection. The overall architecture is shown in Figure 3. Figure 3: Overall architecture Workflow: The sensors in the factory are sending time series measurements to Azure IoT Hub. These sensors are measuring for multiple machines: Product Type Air Temperature (°C) Process Temperature (°C) Rotational Speed Torque Tool Wear (min) IoT Hub will feed these sensor data to Azure Stream Analytics, where the data will be filtered and pushed to MongoDB Atlas time series collections. The functionality of Stream Analytics can be extended by implementing machine learning models to do real-time predictive analytics on streaming input data. The prediction results can also be stored in MongoDB in a separate collection. The sensor data contains the device_id field which helps us filter data coming from different machines. As MongoDB is a document database, we do not need to create multiple tables to store this data, in fact we can just use one collection for all the sensor data coming from various devices or machines. Once the data is received in MongoDB, sum and mean values of sensor data will be calculated for the predefined production shift duration and the results will be pushed to MongoDB Atlas Charts for visualization. MongoDB Time series window functions are used in an aggregation pipeline to produce the desired result. When a machine stoppage or breakdown occurs during the course of production, it may lead to downtime because the operator has to find out the cause of the failure before the machine can be put back into production. The sensor data collected from the machines can be used to train a machine learning model that can automatically predict the root cause when a failure occurs and significantly reduce the time spent on manual root cause finding on the shop floor. This can lead to increased availability of machines and thus more production time per shift. To achieve this goal, our first task is to identify the types of failures we want to predict. We can work with the machine owners and operators to identify the most common failure types and note that down. With this important step completed, we can identify the data sources that have relevant data about that failure type. If need be, we can update the Stream Analytics filter as well. Once the right data is identified, we train a Decision Tree Classifier model in Azure Machine Learning and deploy it as a binary value as a separate collection inside MongoDB. Atlas Scheduled Triggers are used to trigger the model (via an Azure Function) and the failure prediction results are written back results into a separate Failures collection in MongoDB. Scheduled triggers’ schedule can be aligned to production schedule so that it only fires when a changeover occurs for example. After a failure is detected, the operator and supervisor needs to be notified immediately. Using App Services, a mobile application is developed to send notifications and alerts to floor supervisor and machine operator once a failure root cause is predicted. Figure 4 shows the mobile app user interface where the user has an option to acknowledge the alert. Thanks to Atlas Device Sync , even when the mobile device is facing unreliable connectivity, the data keeps in sync between Atlas cluster and Realm database in the app. MongoDB’s Realm , is an embedded database technology already used on millions of mobile phones as part of mobile apps as well as infotainment like systems. FIgure 4: Alert app user interface Business benefits of using MongoDB Atlas as smart manufacturing data hub Scalability: MongoDB is a highly scalable document based database that can handle large amounts of structured, semi-structured and unstructured data. Native time series collections are available that help with storing large amounts of data generated by IIoT enabled equipment in a highly compressed manner. Flexibility: MongoDB stores data in a flexible, JSON-like format, which makes it easy to store and query data in a variety of ways. This flexibility makes it well-suited for handling the different data structures needed to store sensor data, ML models and prediction results, all in one database. This removes the need for maintaining separate databases for each type of data reducing IT sprawl in manufacturing organizations. Real-time Analytics: As sensor data comes in, MongoDB aggregation pipelines can help in generating features to be used for machine learning models. Atlas Charts can be set up in minutes to visualize important features and their trends in near real time. BI Analytics: Analysts can use the Atlas SQL interface to access MongoDB data from SQL based tools. This allows them to work with rich, multi-structured documents without defining a schema or flattening data. In a connected factory setting, this can be useful to generate reports for failures over a period of time and comparison between different equipment failures types. Data can be blended from MongoDB along with other sources of data to provide a 360 degree view of production operations. Faster Mobile Application Development: Atlas device sync bidirectionally connects and synchronizes Realm databases inside mobile applications with the MongoDB Atlas backend, leading to faster mobile application development and less time needed for maintenance of deployed applications. Conclusion The MongoDB Atlas developer data platform is designed and engineered to help speed up your journey towards smart manufacturing. It is not just suitable for high speed time series workloads but also for workloads that power mobile applications and BI Dashboards – leading to smarter applications, increased productivity and eventually smarter factories. Learn more All the source code used in this project, along with a detailed deployment guide, is available on our public Github page . To learn more about how MongoDB enables IIoT for our customers, please visit our IIoT use cases page . Get started today with MongoDB Atlas on Azure Marketplace listing .

February 27, 2023

Simplifying IoT Connectivity with myDevices and MongoDB

In the highly competitive era of Industry 4.0, companies that are able to adopt emerging Internet of Things (IoT) technologies and shift from traditional offerings to digitally differentiated ones are moving to the forefront of their respective industries. McKinsey & Company estimates that by 2030, IoT could enable $5.5 trillion to $12.6 trillion in value globally, including the value captured by consumers and customers of IoT products and services. From smart thermostats to smart factories, IoT already connects billions of devices worldwide. Figure 1 shows potential areas where IoT solutions make a difference. Figure 1:   IoT applications by industry (non-exhaustive). All of these IoT applications and solutions require technologies that can offer low-power operation, low-cost, and low complexity in setting up and maintaining end devices. End devices that are able to communicate wirelessly over large distances with low-power consumption are key. The data generated by IoT devices is time series and high frequency, placing a unique strain on the underlying data infrastructure. Because of the polymorphic nature of IoT sensor data, the database must support flexible data schemas, making it easy for developers to work with the data. It must also ensure that the IoT applications are resilient to future changes. MongoDB embraces the variety and volume of IoT data without compromising on performance. Through its document model, MongoDB eliminates data movement and blends time series with the rest of the enterprise data in a single developer data platform. In this article, we’ll describe how myDevices leverages the MongoDB developer data platform for IoT. Overview of myDevices myDevices is a U.S.-based IoT solutions company that empowers system integrators, MSPs, ISVs, VARS, and enterprise customers to quickly deploy IoT solutions to their customers. The company has more than 1000 plug and play sensors and multiple Long Range Wide Area Network (LoRaWAN) gateway options to create IoT solutions for a variety of use cases. Over time, myDevices has created the world’s most extensive IoT device catalog from more than 150 hardware manufacturers around the globe. LoRaWAN offers unique IoT benefits, such as long range and coverage, which may reach up to 15 kilometers in line of sight (LOS). It offers ultra-low power consumption for end devices, low-cost infrastructure, and high capacity, which makes it possible to link thousands of devices to one single gateway. myDevices understands that connecting devices from disparate manufacturers can be very challenging; thus, they have created a no-code solution that includes plug-and-play templates to connect sensors to the gateway just by scanning a QR code. After the sensor is connected to the gateway, users can perform remote monitoring and device management from a single-view interface. They can also get alerts through text and email and set up charts for visualization of sensor data. The alert rules can be configured as time based or threshold based in the myDevices platform. The myDevices IoT platform is secure from the edge to the application layer through the cloud. The security is composed of LoRaWAN network security at the edge, TLS to the cloud, and SAML at the application layer. Figure 2 shows the architecture of the myDevices platform and how it connects to the sensors. Figure 2:   MyDevices architecture. myDevices also has multiple ready-to-go solutions for a variety of IoT use cases and applications. From machine health predictive maintenance to soil moisture detection, there are sensors that just work with the IoT in a box application. It takes only minutes to set up connectivity between the sensor and myDevices cloud, and myDevices enhances productivity because you don’t have to worry about writing code to extract data from the sensors and establishing secure connectivity with the gateway. As LoRaWAN enables hundreds, if not thousands, of sensors sending data to a single gateway, it requires a database that can easily and automatically scale. When it comes to publishing data out of myDevices cloud to MongoDB Atlas, myDevices provides a webhook integration functionality that can be set up in minutes to establish connectivity between the two systems. Database requirements for IoT and MongoDB Atlas MongoDB and MongoDB Atlas are ideal partners for any IoT deployment, offering: Deployment flexibility (on-premises, in-field, cloud) Multi-cloud flexibility (AWS, Azure, GCP) Schema flexibility (frequent changes and additions) The ability to blend different data (time series, operational) Real-time analytics readiness Automated data tiering As a result, IoT data platforms and service providers, such as Bosch and Software AG, as well as some of the world’s most intensive IoT users, including Toyota, Mercedes-Benz, and Vodafone, choose MongoDB for their IoT platforms and services. MongoDB’s developer data platform supports the entire IoT data life cycle, from ingestion, storage, querying, real-time analytics, and visualization to online archiving (Figure 3). MongoDB Atlas brings the core components of real-time analytics into one developer data platform. Figure 3:   MongoDB Developer Data Platform for IoT. Let's talk about a few features that directly support IoT applications: Native time series platform: MongoDB supports native time series collections with hands-free schema optimization supporting high-efficiency storage and low-latency queries. This is an extremely important feature for IoT applications. Change streams: MongoDB change streams allow applications to access real-time data changes in the database without any complexity or risk. IoT applications can use change streams to subscribe to all data changes on a single collection, a database or an entire deployment and immediately react to them. This approach enables quick response time and fast decision making. Aggregation framework: By using the built-in aggregation framework in MongoDB, users are able to do real-time analytics without having to move the data to another platform. By using the aggregation framework, the work is done inside MongoDB, and the final results can be sent to the application, typically resulting in a smaller amount of data being moved around. For IoT applications, this can be a powerful tool to only transmit the filtered data to the Cloud or central storage resulting in improved security and reduced cost. Data Lake: As data is ingested, Atlas Data Lake automatically optimizes and partitions the data in a format and structure best for analytical queries. This capability significantly reduces the complexity of transforming data for the data scientist tasked with building machine learning models for analytical use cases and applications Data Federation: Atlas Data Federation provides the ability to federate queries across data stored in various supported storage formats, including Atlas Clusters, Data Lake Datasets, AWS S3 buckets, and HTTP stores. This feature reduces complexity of bringing data together for analytical model testing purposes. Data API: Companies can use Atlas Data API to integrate Atlas into any apps and services that support HTTPS requests. Leveraging this feature, the data from the myDevices cloud can be sent to Atlas and then used for storage and for analytical purposes using the aggregation framework or via the Atlas ecosystem connectors with third-party analytical software. Ecosystem integration: MongoDB Spark Connector opens up access to all Spark libraries for use with MongoDB datasets: Datasets for analysis with SQL (benefiting from automatic schema inference), streaming, machine learning, and graph APIs. Charts: MongoDB Charts is the best way to visualize IoT data stored in MongoDB. Charts is built specifically for the document model, no ETL, no time loss to data manipulation or duplication required to visualize rich JSON data. Using Charts, powerful engaging data experiences can be created for the use case stakeholders in no time. Integrating Atlas and myDevices using Webhooks and Data API myDevices offers a variety of no-code integrations for its clients to quickly get started by sending data to the platform of their choice. For MongoDB Atlas clients, this is great news because, by using myDevices Webhook integrator and payload transformation feature, MongoDB Atlas clients can receive and store LoRa sensor data into the specified collection. Let’s run through the methodology to perform this integration: Step 1: Log into your Atlas Cluster and set up Data API and API key. The MongoDB Atlas Data API lets you read and write data in Atlas with standard HTTPS requests. To use the Data API, all you need is an HTTPS client and a valid API key. It is important to understand that the Data API is not a direct connection to the MongoDB database. Instead, it routes requests through a fully managed middleware layer, called Atlas App Services, that sits between your cluster and client apps. This layer handles user authentication and enforces data access rules to ensure that the data is secure. The Data API supports two types of endpoints: Data API endpoints are automatically generated endpoints that each represent a MongoDB operation. You can use the endpoints to create, read, update, delete, and aggregate documents in a MongoDB data source. Custom endpoints are app-specific API routes handled by functions that you write. You can use custom endpoints to run your app's backend logic or as webhooks that integrate with external services. In this example, we are using a data API endpoint. You can follow these easy steps to enable Data API and create a Data API Key. Step 2: Log in your myDevices Console and set up integrations After you log in, click on new webhook creation through the INTEGRATIONS option on the right-hand panel (Figure 4). For the purpose of this article, we are assuming that you have already created an organization in myDevices and added sensors and gateways to it. If you have not, please refer to myDevices API docs to get started. Figure 4:   Set up integrations in myDevices. Step 3: Click on Webhook integration to open up the new Webhook creation panel. In this step, choose Webhook as the desired integration option, as shown in Figure 5. Figure 5:   Choose Webhook as the integration option. Step 4: Add key information. In this step, you’ll want to include key information, such as Url, which is your Data API endpoint, Webhook Header, which will include the api-key at the very minimum, and the payload transform script, where you can specify the cluster, database, and collection where this sensor data needs to be stored (Figure 6). Figure 6:   Paste the endpoint generated by Data API in Atlas. An example payload transformation script looks like the following. This is according to Data API requirements where you have to specify the cluster, database and collection name in the raw body data. function Transform(event, metadata) { return { dataSource: "my_cluster", database: "my_database", collection: "current_sensor", document: event, }; } Step 5: Save your webhook. Once you save your webhook, you can observe sensor data flowing into your MongoDB Atlas collection from the actual device using MongoDB Compass or Atlas Charts (Figure 7). For more details on how to create Charts, please visit the Atlas Charts documentation . Figure 7: Visualize sensor data using Atlas Charts. Conclusion We have shown how easy it is to connect myDevices IoT platform with MongoDB using the Data API . The overall architecture is shown in Figure 8. Figure 8: End-to-end architecture of myDevices and MongoDB Atlas integration. Simplifying IoT connectivity is of paramount importance for any organization looking to embark on a digital transformation journey. Fortunately, both myDevices and MongoDB Atlas provide platforms that simplify management of the full life cycle of an IoT device from provisioning to connectivity to data storage and archival. To learn more about how MongoDB enables IoT for our customers, please visit our IoT use cases page .

December 6, 2022

Achieving Industrial Connectivity at Scale with Wimera and MongoDB

Thank you to Karolina Ruiz Rogelj for her contributions to this post. Industry 4.0 (I4.0) represents the beginning of the Fourth Industrial Revolution. It includes the current trend of automation technologies in the manufacturing industry as well as disruptive technologies and concepts, such as cyber-physical systems (CPS), Industrial Internet of Things (IIoT), cloud computing, and immersive visualization. Through Industry 4.0, embedded systems, semantic machine-to-machine communication, IIoT, and CPS technologies are integrating the virtual space with the physical world. These technologies are enabling a new generation of industrial systems, such as smart factories, to deal with the complexity of fast-paced and hyper-personalized production. In this article, we’ll explore Wimera’s unique solutions to the challenges of I4.0 and IIoT, built with MongoDB. Information and insights With IIoT, existing industrial systems will be modernized to drive digital transformation and unlock tomorrow's smart enterprise. IIoT has been finding its way into products and sensors while revolutionizing existing manufacturing systems; thus, it is considered a key enabler for the next generation of advanced manufacturing. Industry 4.0 generally comprises many complex components and has broad applications in all manufacturing sectors. The first challenge faced by manufacturing companies when embarking on the I4.0 journey is to sensorize and connect their manufacturing equipment in order to collect, store, and analyze data for information and insights. Wimera Systems is solving this challenge as an I4.0 enablement company offering IIoT solutions using their unique hardware, software application, and AI/ML-based analytics engine. Wimera’s Smart Factory Suite has seen tremendous growth, with 2500+ global installations across 50+ customers. MongoDB has been pivotal to that growth, acting as the core component of the IIoT suite and enabling the company to offer its services at scale without having to worry about managing the complexity of an IIoT database. Bringing AI-powered IIoT to the manufacturing shop floor Manufacturing companies are emerging from the pandemic with a renewed focus on digital transformation and smart factories investment. COVID-19 has heightened the need for IIoT technology and innovation, forcing manufacturers to compete in a digitalized business environment. Many manufacturers still operate using legacy technologies and systems; on most shop floors, equipment and operator efficiency are manually calculated and tracked using spreadsheets. The machines are maintained using time-based rather than condition-based maintenance strategies. And, no real-time visibility exists on consumables and tools usage. All these practices result in increased maintenance costs, suboptimal production, and ultimately, customer dissatisfaction. Wimera understands these challenges all too well, which is why they created the Smart Factory Suite supporting both on-premise and cloud deployments. The Smart Factory Suite provides insights for managing the entire production landscape through interconnected devices and machines, operations, and facilities. It can predict and make real-time adjustments for increased production efficiency and less downtime. The suite is primarily utilized for empowering manufacturing operations, equipment maintenance, warehouse operations, and inventory management. With Smart Factory Suite, Wimera serves a wide range of manufacturing industry sectors including, but not limited to, automotive, electronics, chemical, and food processing companies. Deploy and run anywhere with MongoDB MongoDB, with its freedom to run anywhere, lets Wimera offer both on-premises and cloud deployment options for its customers. In both cases, the suite is directly connected with machine controllers using Wimera libraries for all popular Programmable Logic Controller (PLC) brands. The suite is also connected to legacy machines through external sensors installed by the Wimera team. Data is extracted via the Wimera ReMON Data Acquisition (DAQ) device (Figure 1) that utilizes the MongoDB database as the persistent data storage. MongoDB’s flexible data model makes it easy to combine and enrich this data and enables live dashboards and instant alerts for factory personnel. The data collected and optimized by ReMON DAQ is further fed to ReMON AI , an advanced analytics engine. ReMON AI provides advanced analytics through AI/ML models and leverages MongoDB to deliver application-driven analytics in real time. Figure 1: ReMON DAQ and ReMON AI (source: Wimera ReMON ). Whether through on-premises or cloud deployment (Figures 2 and 3), Wimera’s customers have benefited from MongoDB’s capabilities that are critical for IIoT applications, such as time series collections and the flexible, intuitive document data model. Figure 2: Wimera IoT architecture on premises. Figure 3: Wimera IoT architecture on cloud (using MongoDB on AWS). In one customer example, while deploying IIoT at a multinational CNC machine shop, the customer preferred to use their existing production monitoring application enriched with IoT data coming from Wimera’s Smart Factory Suite. In this case, MongoDB enabled easy and seamless integration of the IoT application with the customer's application via a simple API. Additionally, high-speed data coming from a vibration sensor was handled effectively by MongoDB time series collections, resulting in real-time alerts sent to maintenance teams for instant corrective actions on the shop floor. In another example, a multinational automotive manufacturer wanted a single platform that could collect and combine data coming from vendors in different formats and contexts. MongoDB's flexible document model helped manage the varied data types easily, allowing the customer to benefit from a single application capable of managing multiple vendors in parallel. This flexibility offered by MongoDB enables the customer to keep adding new vendors instantly without changing the underlying cloud infrastructure or tweaking schemas. Interested readers can check out additional case studies on Wimera’s website. Building better together Wimera and MongoDB’s partnership gives customers confidence with validated architectures to ensure successful, optimized, and scalable deployments at their facilities. Wimera’s continued partnership with MongoDB also helps guide the company’s product roadmap as we expand in the IIoT, Smart Factory market together. MongoDB is the only enterprise grade database chosen by the Wimera development team due to easy handling of the large volume of data generated from machines and sensors while maintaining a high performance… If we want to insert thousands of records in a second, then MongoDB is the best choice for that given our solutions are for Industrial IoT. Also, horizontal scaling (adding new columns) is not an easy process in any RDBMS system. But in the case of MongoDB, it is very easy Nagarajan Narayanasamy, CEO, Wimera Systems Private Limited A bright future ahead Since 2019, Wimera has been an early adopter of MongoDB for their Industrial IoT application for discrete manufacturing industries and process industries on multiple domains. “Currently, Narayanasamy says, “Wimera’s Industrial IoT solutions are matured, and we are focused on scaling globally.” Wimera now targets expansion in India, APAC, EU, and USA for the discrete manufacturing and process industries and also for select OEMs and machine builders. “As MongoDB continues to scale itself globally through its multi-cloud data distribution strategy, we see a good synergy partnering with MongoDB for the mutual benefit of both companies and the community as a whole. We also would like to work with MongoDB on the technology roadmap and solve some of the real-life challenges faced by manufacturing industries,” Narayanasamy says. Wimera has recently started their MongoDB Atlas journey, and the adoption will grow as their customers demand more cloud solutions compared to current on-premises deployments. MongoDB will continue to help IoT companies like Wimera take their product offering to the next level and enable their customers to digitally transform their manufacturing operations. To learn more about MongoDB’s role in industrial connectivity and IIoT, please visit our Manufacturing and Industrial IoT page.

December 1, 2022