Humza Akhtar

19 results

Next-Generation Mobility Solutions with Agentic AI and MongoDB Atlas

Driven by advancements in vehicle connectivity, autonomous systems, and electrification, the automotive and mobility industry is currently undergoing a significant transformation. Vehicles today are sophisticated machines, computers on wheels, that generate massive amounts of data, driving demand for connected and electric vehicles. Automotive players are embracing artificial intelligence (AI), battery electrical vehicles (BEVs), and software-defined vehicles (SDVs) to maintain their competitive advantage. However, managing fleets of connected vehicles can be a challenge. As cars get more sophisticated and are increasingly integrated with internal and external systems, the volume of data they produce and receive greatly increases. This data needs to be stored, transferred, and consumed by various downstream applications to unlock new business opportunities. This will only grow: the global fleet management market is projected to reach $65.7 billion by 2030, growing at a rate of almost 10.8% annually. A 2024 study conducted by Webfleet showed that 32% of fleet managers believe AI and machine learning will significantly impact fleet operations in the coming years; optimizing route planning and improving driver safety are the two most commonly cited use cases. As fleet management software providers continue to invest in AI, the integration of agentic AI can significantly help with things like route optimization and driver safety enhancement. For example, AI agents can process real-time traffic updates and weather conditions to dynamically adjust routes, ensuring timely deliveries while advising drivers on their car condition. This proactive approach contrasts with traditional reactive methods, improving vehicle utilization and reducing operational and maintenance costs. But what are agents? In short, they are operational applications that attempt to achieve goals by observing the world and acting upon it using the data and tools the application has at its disposal. The term "agentic" denotes having agency, as AI agents can proactively take steps to achieve objectives without constant human oversight. For example, rather than just reporting an anomaly based on telemetry data analysis, an agent for a connected fleet could autonomously cross-check that anomaly against known issues, decide whether it's critical or not, and schedule a maintenance appointment all on its own. Why MongoDB for agentic AI Agentic AI applications are dynamic by nature as they require the ability to create a chain of thought, use external tools, and maintain context across their entire workflow. These applications generate and consume diverse data types, including structured and unstructured data. MongoDB’s flexible document model is uniquely suited to handle both structured and unstructured data as vectors. It allows all of an agent’s context, chain-of-thought, tools metadata, and short-term and long-term memory to be stored in a single database. This means that developers can spend more time on innovation and rapidly iterate on agent designs without being constrained by rigid schemas of a legacy relational database. Figure 1. Major components of an AI agent. Figure 1 shows the major components of an AI agent. The agent will first receive a task from a human or via an automated trigger, and will then use a large language model (LLM) to generate a chain of thought or follow a predetermined workflow. The agent will use various tools and models during its run and store/retrieve data from a memory provider like MongoDB Atlas . Tools: The agent utilizes tools to interact with the environment. This can contain API methods, database queries, vector search, RAG application, anything to support the model Models: can be a large language model (LLM), vision language model (VLM), or a simple supervised machine learning model. Models can be general purpose or specialized, and agents may use more than one. Data: An agent requires different types of data to function. MongoDB’s document model allows you to easily model all of this data in one single database. An agentic AI spans a wide range of functional tools and context. The underlying data structures evolve throughout the agentic workflow and as an agent uses different tools to complete a task. It also builds up memory over time. Let us list down the typical data types you will find in an agentic AI application. Data types: Agent profile: This contains the identity of the agent. It includes instructions, goals and constraints. Short-term memory: This holds temporary, contextual information—recent data inputs or ongoing interactions—that the agent uses in real-time. For example, short-term memory could store sensor data from the last few hours of vehicle activity. In certain agentic AI frameworks like Langgraph, short term memory is implemented through a checkpointer. The checkpointer stores intermediate states of the agent’s actions and/or reasoning. This memory allows the agent to seamlessly pause and resume operations. Long-term memory: This is where the agent stores accumulated knowledge over time. This may include patterns, trends, logs and historical recommendations and decisions. By storing each of these data types into rich, nested documents in MongoDB, AI developers can create a single-view representation of an agent’s state and behavior. This enables fast retrieval and simplifies development. In addition to the document model advantage, building agentic AI solutions for mobility requires a robust data infrastructure. MongoDB Atlas offers several key advantages that make it an ideal foundation for these AI-driven architectures. These include: Scalability and flexibility: Connected Car platforms like fleet management systems need to handle extreme data volumes and variety. MongoDB Atlas is proven to scale horizontally across cloud clusters, letting you ingest millions of telemetry events per minute and store terabytes of telemetry data with ease. For example, the German company ZF uses MongoDB to process 90,000 vehicle messages per minute (over 50 GB of data per day) from hundreds of thousands of connected cars. The flexibility of the document model accelerates development and ensures your data model stays aligned with the real-world entities it represents. Built-in vector search: AI agents require a robust set of tools to work with. One of the most widely used tools is vector search, which allows agents to perform semantic searches on unstructured data like driver logs, error codes descriptions, and repair manuals. MongoDB Atlas Vector Search allows you to store and index high-dimensional vectors alongside your documents and to perform semantic search over unstructured data. In practice, this means your AI embeddings live right next to the relevant vehicle telemetry and operational data in the database, simplifying architectures for use cases like the connected car incident advisor, in which a new issue can be matched against past issues before passing contextual information to the LLM. For more, check out this example of how an automotive OEM leverages vector search for audio based diagnostics with MongoDB Atlas Vector Search. Time series collections and real-time data processing: MongoDB Atlas is designed for real-time applications. It provides time series collections for connected car telemetry data storage, change streams, and triggers that can react to new data instantly. This is crucial for agentic AI feedback loops, where ongoing data ingestion and learning are happening continuously. Best-in-class embedding models with Voyage AI: In early 2025, MongoDB acquired Voyage AI , a leader in embedding and reranking models. Voyage AI embedding models are currently being integrated into MongoDB Atlas, which means developers will no longer need to manage external embedding APIs, standalone vector stores, or complex search pipelines. AI retrieval will be built into the database itself, making semantic search, vector retrieval, and ranking as seamless as traditional queries. This will reduce the time required for developing agentic AI applications. Agentic AI in action: Connected fleet incident advisor Figure 2 shows a list of use cases in the Mobility sector, sorted by various capabilities that an agent might demonstrate. AI agents excel at managing multi-step tasks via context management across tasks, they automate repetitive tasks better than Robotic process automation (RPA), and they demonstrate human-like reasoning by revisiting and revising past decisions. These capabilities enable a wide range of applications both during the manufacturing of a vehicle and while it's on the road, connected and sending telemetry. We will review a use case in detail below, and will see how it can be implemented using MongoDB Atlas, LangGraph, Open AI, and Voyage AI. Figure 2. Major use cases of agentic AI in the mobility and manufacturing sectors. First, the AI agent connects to traditional fleet management software and supports the fleet manager in diagnosing and advising the drivers. This is an example of a multi-step diagnostic workflow that gets triggered when a driver submits a complaint about the vehicle's performance (for example, increased fuel consumption). Figure 3 shows the sequence diagram of the agent. Upon receiving the driver complaint, it creates a chain of thought that follows a multi-step diagnostic workflow where the system ingests vehicle data such as engine codes and sensor readings, generates embeddings using the Voyage AI voyage-3-large embedding model, and performs a vector search using MongoDB Atlas to find similar past incidents. Once relevant cases are identified, those–along with selected telemetry data–are passed to OpenAI gpt-4o LLM to generate a final recommendation for the driver (for example, to pull off immediately or to keep driving and schedule regular maintenance). All data, including telemetry, past issues, session logs, agent profiles, and recommendations are stored in MongoDB Atlas, ensuring traceability and the ability to refine diagnostics over time. Additionally, MongoDB Atlas is used as a checkpointer by LangGraph, which defines the agent's workflow. Figure 3. Sequence diagram for a connected fleet advisor agentic workflow. Figure 4 shows the agent in action, from receiving an issue to generating a recommendation. So by leveraging MongoDB’s flexible data model and powerful Vector Search capabilities, we can agentic AI can transform fleet management through predictive maintenance and proactive decision-making. Figure 4. The connected fleet advisor AI agent in action. To set up the use case shown in this article, please visit our GitHub repository . And to learn more about MongoDB’s role in the automotive industry, please visit our manufacturing and automotive webpage . Want to learn more about why MongoDB is the best choice for supporting modern AI applications? Check out our on-demand webinar, “ Comparing PostgreSQL vs. MongoDB: Which is Better for AI Workloads? ” presented by MongoDB Field CTO, Rick Houlihan.

April 4, 2025

Multi-Agent Collaboration for Manufacturing Operations Optimization

While there are some naysayers across the media landscape who doubt the potential impact of AI innovations, for those of us immersed in implementing AI on a daily basis, there’s wide agreement that its potential is huge and world-altering. It’s now generally accepted that Large Language Models (LLMs) will eventually be able to perform tasks as well—if not better—than a human. And the size of the potential AI market is truly staggering. Bain’s AI analysis estimates that the total addressable market (TAM) for AI and gen AI-related hardware and software will grow between 40% and 55% annually, reaching between $780 billion and $990 billion by 2027. This growth is especially relevant to industries like manufacturing, where generative AI can be applied across the value chain. From inventory categorization to product risk assessments, knowledge management, and predictive maintenance strategy generation, AI's potential to optimize manufacturing operations cannot be overstated. But in order to realize the transformative economic potential of AI, applications powered by LLMs need to evolve beyond chatbots that leverage retrieval-augmented generation (RAG). Truly transformative AI-powered applications need to be objective-driven, not just responding to user queries but also taking action on behalf of the user. This is crucial in complex manufacturing processes. In other words, they need to act like agents. Agentic systems, or compound AI systems, are currently emerging as the next frontier of generative AI applications. These systems consist of a single or multiple AI agents that collaborate with each other and use tools to provide value. An AI agent is a computational entity containing short- and long-term memory, which enables it to provide context to an LLM. It also has access to tools, such as web search and function calling, that enable it to act upon the response from an LLM or provide additional information to the LLM. Figure 1. Basic components of an agentic system. An agentic system can have more than one AI agent. In most cases, AI agents may be required to interact with other agents within the same system or external systems., They’re expected to engage with humans for feedback or review of outputs from execution steps. AI agents can also comprehend the context of outputs from other agents and humans, and change their course of action and next steps. For example, agents can monitor and optimize various facets of manufacturing operations simultaneously, such as supply chain logistics and production line efficiency. There are certain benefits of having a multi-agent collaboration system instead of having one single agent. You can have each agent customized to do one thing and do it well. For example, one agent can create meeting minutes while another agent writes follow-up emails. It can also be implemented on predictive maintenance, with one agent analyzing machine data to find mechanical issues before they occur while another optimizes resource allocation, ensuring materials and labor are utilized efficiently. You can also provision dedicated resources and tools for different agents. For example, one agent uses a model to analyze and transcribe videos while the other uses models for natural language processing (NLP) and answering questions about the video. Figure 2. Multi-agent collaboration system. MongoDB can act as the memory provider for an agentic system. Conversation history alongside vector embeddings can be stored in MongoDB leveraging the flexible document model. Atlas Vector Search can be used to run semantic search on stored vector embeddings, and our sharding capabilities allow for horizontal scaling without compromising on performance. Our clients across industries have been leveraging MongoDB Atlas for their generative AI use cases , including agentic AI use cases such as Questflow , which is transforming work by using multi-agent AI to handle repetitive tasks in strategic roles. Supported by MiraclePlus and MongoDB Atlas, it enables startups to automate workflows efficiently. As it expands to larger enterprises, it aims to boost AI collaboration and streamline task automation, paving the way for seamless human-AI integration. The concept of a multi-agent collaboration system is new, and it can be challenging for manufacturing organizations to identify the right use case to apply this cutting-edge technology. Below, we propose a use case where three agents collaborate with each other to optimize the performance of a machine. Multi-agent collaboration use case in manufacturing In manufacturing operations, leveraging multi-agent collaboration for predictive maintenance can significantly boost operational efficiency. For instance, consider a production environment where three distinct agents—predictive maintenance, process optimization, and quality assurance—collaborate in real-time to refine machine operations and maintain the factory at peak performance. In Figure 3, the predictive maintenance agent is focused on machinery maintenance. Its main tasks are to monitor equipment health by analyzing sensor data generated from the machines. It predicts machine failures and recommends maintenance actions to extend machinery lifespan and prevent downtime as much as possible. Figure 3. A multi-agent system for production optimization. The process optimization agent is designed to enhance production efficiency. It analyzes production parameters to identify inefficiencies and bottlenecks, and it optimizes said parameters by adjusting them (speed, vibration, etc.) to maintain product quality and production efficiency. This agent also incorporates feedback from the other two agents while making decisions on what production parameter to tune. For instance, the predictive maintenance agent can flag an anomaly in a milling machine temperature sensor reading; for example, if temperature values are going up, the process optimization agent can review the cutting speed parameter for adjustment. The quality assurance agent is responsible for evaluating product quality. It analyzes optimized production parameters and checks how those parameters can affect the quality of the product being fabricated. It also provides feedback for the other two agents. The three agents constantly exchange feedback with each other, and this feedback is also stored in the MongoDB Atlas database as agent short-term memory. In contrast, vector embeddings and sensor data are persisted as long-term memory. MongoDB is an ideal memory provider for agentic AI use case development thanks to its flexible document model, extensive security and data governance features, and horizontal scalability. All three agents have access to a "search_documents" tool, which leverages Atlas Vector Search to query vector embeddings of machine repair manuals and old maintenance work orders. The predictive maintenance agent leverages this tool to figure out additional insights while performing machine root cause diagnostics. Set up the use case shown in this article using our repo . To learn more about MongoDB’s role in the manufacturing industry, please visit our manufacturing and automotive webpage . To learn more about AI agents, visit our Demystifying AI Agents guide .

February 19, 2025

Saving Energy, Smarter: MongoDB and Cedalo for Smart Meter Systems

The global energy landscape is undergoing a significant transformation, with energy consumption rising 2.2% in 2023, surpassing the 2010-2019 average of 1.5% per year. This increase is largely due to global developments in BRICS member countries—Brazil, Russia, India, China, and South Africa. As renewable sources like solar power and wind energy become more prevalent (in the EU, renewables accounted for over 50% of the power mix in the first quarter of 2024 ), ensuring a reliable and efficient energy infrastructure is crucial. Smart meters, the cornerstone of intelligent energy networks, play a vital role in this evolution. According to IoT analyst firm Berg Insight, the penetration of smart meters is skyrocketing, with the US and Canada expected to reach nearly 90% adoption by 2027, whereas China is expected to account for as much as 70–80% of smart electricity meter demand across Asia in the next few years. This surge is indicative of a growing trend towards smarter, more sustainable energy solutions. In Central Asian countries, the Asian Development Bank is supporting the fast deployment of smart meters to save energy and improve the financial position of countries' power utilities. This article will delve into the benefits of smart meters, the challenges associated with managing their data, and the innovative solutions offered by MongoDB and Cedalo. The rise of smart meters Smart meters, unlike traditional meters that require manual readings, collect and transmit real-time energy consumption data directly to energy providers. This digital transformation offers numerous benefits, including: Accurate Billing: Smart meters eliminate the need for estimations, ensuring that consumers are billed precisely for the energy they use. Personalized Tariffs: Energy providers can offer tailored tariffs based on individual consumption patterns, allowing consumers to take advantage of off-peak rates, special discounts, and other cost-saving opportunities. Enhanced Grid Management: Smart meter data enables utilities to optimize grid operations, reduce peak demand, and improve overall system efficiency. Energy Efficiency Insights: Consumers can gain valuable insights into their energy usage patterns, identifying areas for improvement and reducing their overall consumption. With the increasing adoption of smart meters worldwide, there is a growing need for effective data management solutions to harness the full potential of this technology. Data challenges in smart meter adoption Despite the numerous benefits, the widespread adoption of smart meters also presents significant data management challenges. To use smart metering, power utility companies need to deploy a core smart metering ecosystem that includes the smart meters themselves, the meter data collection network, the head-end system (HES), and the meter data management system (MDMS). Smart meters collect data from end consumers and transmit it to the data aggregator via the Local Area Network (LAN). The transmission frequency can be adjusted to 15 minutes, 30 minutes, or hourly, depending on data demand requirements. The aggregator retrieves the data and then transmits it to the head-end system. The head-end system analyzes the data and sends it to the MDMS. The initial communications path is two-way, signals or commands can be sent directly to the meters, customer premise, or distribution device. Figure 1: End-to-end data flow for a smart meter management system / advanced metering infrastructure (AMI 2.0) When setting up smart meter infrastructure, power, and utility companies face several significant data-related challenges: Data interoperability: The integration and interoperability of diverse data systems pose a substantial challenge. Smart meters must be seamlessly integrated with existing utility systems and other smart grid components often requiring extensive upgrades and standardization efforts. Data management: The large volume of data generated by smart meters requires advanced data management and analytics capabilities. Utilities must implement robust data storage, processing, and analysis solutions to handle real-time time series data streams storage, analysis for anomaly detection, and trigger decision-making processes. Data privacy: Smart meters collect vast amounts of sensitive information about consumer energy usage patterns, which must be protected against breaches and unauthorized access. Addressing these challenges is crucial for the successful deployment and operation of smart meter infrastructure. MQTT: A cornerstone of smart meter communication MQTT , a lightweight publish-subscribe protocol, shines in smart meter communication beyond the initial connection. It's ideal for resource-constrained devices on low-bandwidth networks, making it perfect for smart meters. While LoRaWAN or PLC handle meter-to-collector links, MQTT bridges Head-End Systems (HES) and Meter Data Management Systems (MDMS). Its efficiency, reliable delivery, and security make it well-suited for large-scale smart meter deployments. Cedalo MQTT platform and MongoDB: A powerful combination Cedalo , established in 2017, is a leading German software provider specializing in MQTT solutions. Their flagship product, the Cedalo MQTT Platform, offers a comprehensive suite of features, including the Pro Mosquitto MQTT broker and Management Center . Designed to meet the demands of large enterprises, the platform delivers high availability, audit trail logging, persistent queueing, role-based access control, SSO integration, advanced security, and enhanced monitoring. To complement the platform's capabilities, MongoDB's Time Series collections provide a robust and optimized solution for storing and analyzing smart meter data. These collections leverage a columnar storage format and compound secondary indexes to ensure efficient data ingestion, reduced disk usage, and rapid query processing. Additionally, window functions enable flexible time-based analysis, making them ideal for IoT and analytical applications. Figure 2: MongoDB as the main database for the meter data management system where it receives meter data via Pro Mosquitto MQTT broker. Let us revisit Figure 1 and leverage both the Cedalo MQTT Platform and MongoDB in our design. In Figure 2, the Head-end System (HES) can use MQTT to filter, aggregate, and convert data before storing it in MongoDB. This data flow can be established using the MongoDB Bridge plugin provided by Cedalo. Since the MQTT payload is JSON, it is ideal to store it in MongoDB as the database stores data in BSON (Binary JSON). The MongoDB Bridge plugin offers advanced features such as flexible data import settings (specifying target databases and collections, choosing authentication methods, and selecting specific topics and message fields to import) and advanced collection mapping (mapping multiple MQTT topics to one or more collections with the ability to choose specific fields for insertion). MongoDB's schema flexibility is crucial for adapting to the ever-changing structures of MQTT payloads. Unlike traditional databases, MongoDB accommodates shifts in data format seamlessly, eliminating the constraints of rigid schema requirements. This helps with interoperability challenges faced by utility companies. Once the data is stored in MongoDB, it can be analyzed for anomalies. Anomalies in smart meter data can be identified based on various criteria, including sudden spikes or drops in voltage, current, power, or other metrics that deviate significantly from normal patterns. Here are some common types of anomalies that we might look for in smart meter data: Sudden spikes or drops: These include voltage, current, or power spikes or drops. A sudden increase or decrease in voltage beyond expected limits. Outliers: Data points that are significantly different from the majority of the data. Unusual patterns: Unusually high or low energy consumption compared to historical data or inconsistent power factor readings. Frequency anomalies: Frequency readings that deviate from the normal range. MongoDB's robust aggregation framework can aid in anomaly detection. Both anomaly data and raw data can be stored in time series collections, which offer reduced storage footprint and improved query performance due to an automatically created clustered index on timestamp and _id. The high compression offered addresses the challenge of data management at scale. Additionally, data tiering capabilities like Atlas Online Archive can be leveraged to push cold data into cost-effective storage. MongoDB also provides built-in security controls for all your data, whether managed in a customer environment or MongoDB Atlas, a fully managed cloud service. These security features include authentication, authorization, auditing, data encryption (including Queryable Encryption ), and the ability to access your data security with dedicated clusters deployed in a unique Virtual Private Cloud (VPC). End-to-end solution Figure 3: End-to-end data flow Interested readers can clone this repository and set up their own MongoDB-based smart meter data collection and anomaly detection solution. The solution follows the pattern illustrated in Figure 3, where a smart meter simulator generates raw data and transmits it via an MQTT topic. A Mosquitto broker receives these messages and then stores them in a MongoDB collection using the MongoDB Bridge. By leveraging MongoDB change streams , an algorithm can retrieve these messages, transform them according to MDMS requirements, and perform anomaly detection. The results are stored in a time series collection using a highly compressed format. The Cedalo MQTT Platform with MongoDB offers all the essential components for a flexible and scalable smart meter data management system, enabling a wide range of applications such as anomaly detection, outage management, and billing services. This solution empowers power distribution companies to analyze trends, implement real-time monitoring, and make informed decisions regarding their smart meter infrastructure. We are actively working with our clients to solve IoT challenges. Take a look at our Manufacturing and Industrial IoT page for more stories.

September 4, 2024

Mobile and Edge Solutions with MongoDB and Ditto

Mobile and edge solutions offer impressive opportunities for profit and growth for a variety of businesses around the world. Companies have consistently found ways to use mobile applications to grow revenue, cut costs, and stay ahead of the competition. In the power and utilities sector, for example, field workers can get enabled quickly by accessing their daily tasks on mobile devices and, in retail, consumers can use mobile apps to skip lines, providing businesses with upselling opportunities that can result in larger transactions. Indeed, mobile commerce is estimated to make up 44.6% of total US retail ecommerce sales in 2024. For banks, increased use of mobile applications can reduce operating costs by decreasing the demand for in-person and phone-based customer service. At the same time, having a mobile app allows financial institutions to reach additional customers, as many internet users around the world (particularly in developing countries) rely on mobile access. Time and time again, we’ve seen that the most successful apps are those thats meet modern user expectations. Specifically, apps need to be fast and reactive, without lags or crashes. And if internet connectivity drops, the app should continue functioning normally until connectivity is restored. In cases where the workforce is located in low-connectivity areas—e.g., warehouses, factories, and rural areas—peer-to-peer sync is a requirement for apps to communicate with each other and sync data. In such an ever-important space, partnerships are critical to combining the strengths of organizations to create solutions that would be challenging to develop independently. At MongoDB, we’re laser-focused on bringing the best solutions to customers. So we’re thrilled to announce MongoDB’s partnership with Ditto , a company that enables consistently fast data synchronization between devices like mobile phones and point-of-sale systems for mission-critical enterprise apps regardless of environment connectivity and existing infrastructure. With MongoDB and Ditto, businesses can drive consistent revenue at the edge without Wi-Fi, servers, or a cloud connection. Retailers can sell products, banks can deliver services, and energy companies can conduct operations anywhere without worrying about connectivity. Welcome to our mobile partner, Ditto Based in San Francisco, Ditto is revolutionizing the mobile app development space. Ditto technology uses existing devices like phones and tablets to create a distributed wireless network that can sync data anytime, even without the internet, Wi-Fi, or servers. With Ditto’s SDK, devices automatically discover, connect, and sync with each other in peer-to-peer (P2P) mesh networks. This means that when the internet goes down or Wi-Fi is spotty, deskless workers can continue to serve customers or complete business-critical workflows. Ditto manages a mesh network of devices and automatically syncs data changes locally in the mesh and opportunistically with the cloud when available. Depending on the environment and device positioning, Ditto intelligently switches between LAN, BLE, P2P Wi-Fi, IP-based transports, and cellular to ensure that apps get the fastest sync. Ditto’s platform has two major components: Small Peer/Ditto SDK: This is the Ditto SDK embedded into an application that lives on a mobile device, point of sale system, IoT device, and more. There can be many Small Peers in a solution. Small Peers self-organize and sync with each other regardless of internet connectivity and with the cloud when connectivity is available. Big Peer: Ditto’s middleware platform that receives the data from small peers and forwards them to MongoDB. And some of the unique value propositions that Ditto offers include: Self-organizing mesh networking: Devices running Ditto-powered apps automatically and securely discover nearby peers and form wireless, distributed networks. Intelligent peer-to-peer data sync: Devices in the mesh exchange data in real-time via Bluetooth Low Energy, Peer-to-Peer Wi-Fi, Local Area Network, and more. Conflict-free replicated data types (CRDTs): Ditto peers each have a local database. To ensure low-bandwidth usage and concurrent edits, only the deltas, or changes, are synced Distributed architecture: As the image below shows, Ditto isn’t reliant on a centralized system to synchronize data. Each device has an embedded database capable of reading, writing, and syncing deltas within the mesh. This means there is no single point of failure, such as a cloud or server. With MongoDB and Ditto working together, developers can create robust data pipelines from mobile to cloud. MongoDB Atlas is a multi-cloud modern database that gives users the versatility they need to build a wide variety of applications—including mobile applications. With MongoDB Atlas, users can scale their mobile applications’ backend confidently with a foundation built for resilience, performance, and security. Additionally, MongoDB Atlas enables delivering fast and consistent mobile user experiences in any region on AWS, Azure, and Google Cloud—or replicate data across multiple regions and clouds to reach wider audiences and protect against broader outages. Read more about Ditto at our partner catalog page .

September 3, 2024

Building Gen AI-Powered Predictive Maintenance with MongoDB

In today’s fast-evolving industrial landscape, digital transformation has become a necessity. From manufacturing plants to connected vehicles, the push towards predictive maintenance excellence is driving organizations to embrace smarter, more efficient ways of managing operations. One of the most compelling advancements in this domain is predictive maintenance powered by generative AI , a cutting-edge approach that will revolutionize how industries maintain and optimize their equipment. For manufacturers seeking maintenance excellence, a unified data store and a modern database are key enablers. These tools provide the foundation for integrating AI applications that can analyze sensor data, predict failures, and optimize maintenance schedules. MongoDB Atlas is the only multi-cloud modern database available that is designed to streamline and speed up developers' data handling. With MongoDB Atlas, developers can enhance end-to-end value chain optimization through AI/ML, advanced analytics, and real-time data processing, supporting cutting-edge mobile, edge, and IoT applications. In this post, we’ll explore the basics of predictive maintenance and how MongoDB can be used for maintenance excellence. Check out our AI Learning Hub to learn more about building AI-powered apps with MongoDB. Understanding the need for predictive maintenance Predictive maintenance is about anticipating and addressing equipment failures before they occur, ensuring minimal disruption to operations. Traditional maintenance strategies, like time-based or usage-based maintenance, are less effective than predictive maintenance because they don’t account for the varying conditions and complexities of machinery. Unanticipated equipment breakdown can result in line stoppage and substantial throughput losses, potentially leading to millions of dollars in revenue loss. Since the pandemic, many organizations have begun significant digital transformations to improve efficiency and resilience. However, a concerning gap exists between tech adoption and return on investment. While 89% of organizations have begun digital and AI transformations, only 31% have seen the expected revenue lift, and only 25% have realized the expected cost savings. These numbers highlight the importance of implementing new technologies strategically. Manufacturers need to carefully consider how AI can address their specific challenges and then integrate them into existing processes effectively. Predictive maintenance boosts efficiency and saves money Predictive maintenance uses data analysis to identify problems in machines before they fail. This allows organizations to schedule maintenance at the optimal time, maximizing machine reliability and efficiency. Indeed, according to Deloitte , predictive maintenance can lead to a variety of benefits, including: 3-5% reduction in new equipment costs 5-20% increase in labor productivity 15-20% reduction in facility downtime 10-30% reduction in inventory levels 5-20% reduction in carrying costs Since the concept was introduced, predictive maintenance has constantly evolved. We've moved beyond basic threshold-based monitoring to advanced techniques like machine learning (ML) models. These models can not only predict failures but also diagnose the root cause, allowing for targeted repairs. The latest trend in predictive maintenance is automated strategy creation. This involves using AI to not only predict equipment breakdowns but also to generate repair plans, ensuring the right fixes are made at the right time. Generative AI in predictive maintenance To better understand how gen AI can be used to build robust predictive maintenance solutions, let's dig into the characteristics of organizations that have successfully implemented AI. They exhibit common traits across five key areas: Identifying high-impact value drivers and AI use cases: Efforts should be concentrated on domains where artificial intelligence yields maximal utility rather than employing it arbitrarily. Aligning AI strategy with data strategy: Organizations must establish a strong data foundation with a data strategy that directly supports their AI goals. Continuous data enrichment and accessibility: High-quality data, readily available and usable across the organization, is essential for the success of AI initiatives. Empowering talent and fostering development: By equipping their workforce with training and resources, organizations can empower them to leverage AI effectively. Enabling scalable AI adoption: Building a strong and scalable infrastructure is key to unlocking the full potential of AI by enabling its smooth and ongoing integration across the organization. Implementing predictive maintenance using MongoDB Atlas When combined with a robust data management platform like MongoDB Atlas, gen AI can predict failures with remarkable accuracy and suggest optimal maintenance schedules. MongoDB Atlas is the only multi-cloud modern database designed to accelerate and simplify how developers work with data. Developers can power end-to-end value chain optimization with AI/ML, advanced analytics, and real-time data processing for innovative mobile, edge, and IoT applications. MongoDB Atlas offers a suite of features perfectly suited for building a predictive maintenance system, as shown in Figure 1 below. Its ability to handle both structured and unstructured data allows for comprehensive condition monitoring and anomaly detection. Here’s how you can build a generative AI-powered predictive maintenance software using MongoDB Atlas: Machine prioritization: This stage prioritizes machines for the maintenance excellence program using a retrieval-augmented generation (RAG) system that takes in structured and unstructured data related to maintenance costs and past failures. Generative AI revolutionizes this process by reducing manual analysis time and minimizing investment risks. At the end of this stage, the organization knows exactly which equipment or assets are well-suited for sensorization. Utilizing MongoDB Atlas, which stores both structured and unstructured data, allows for semantic searches that provide accurate context to AI models. This results in precise machine prioritization and criticality analysis. Failure prediction: MongoDB Atlas provides the necessary tools to implement failure prediction, offering a unified view of operational data, real-time processing, integrated monitoring, and seamless machine learning integration. Sensors on machines, like milling machines, collect data (e.g., air temperature and torque) and process it through Atlas Stream Processing , allowing continuous, real-time data handling. This data is then analyzed by trained models in MongoDB, with results visualized using Atlas Charts and alerts pushed via Atlas Change Streams to mobile devices, establishing an end-to-end failure prediction system. Repair plan generation: To implement a comprehensive repair strategy, generating a detailed maintenance work order is crucial. This involves integrating structured data, such as repair instructions and spare parts, with unstructured data from machine manuals. MongoDB Atlas serves as the operational data layer, seamlessly combining these data types. By leveraging Atlas Vector Search and aggregation pipelines , the system extracts and vectorizes information from manuals and past work orders. This data feeds into a large language model (LLM), which generates the work order template, including inventory and resource details, resulting in an accurate and efficient repair plan. Maintenance guidance generation: Generative AI is used to integrate service notes and additional information with the repair plan, providing enhanced guidance for technicians. For example, if service notes in another language are found in the maintenance management system, we extract and translate the text to suit our application. This information is then combined with the repair plan using a large language model. The updated plan is pushed to the technician’s mobile app via Atlas Change Streams. The system generates step-by-step instructions by analyzing work orders and machine manuals, ensuring comprehensive guidance without manually sifting through extensive documents. Figure 1: Achieving end-to-end predictive maintenance with MongoDB Atlas modern database In the quest for operational excellence, predictive maintenance powered by generative AI and MongoDB Atlas stands out as a game-changer. This innovative approach not only enhances the reliability and efficiency of industrial operations but also sets the stage for a future where AI-driven insights and actions become the norm. By leveraging the advanced capabilities of MongoDB Atlas, manufacturers can unlock new levels of performance and productivity, heralding a new era of smart manufacturing and connected systems. If you would like to learn more about generative AI-powered predictive maintenance, visit the following resources: [Video] How to Build a Generative AI-Powered Predictive Maintenance Software [Whitepaper] Generative AI in Predictive Maintenance Applications [Whitepaper] Critical AI Use Cases in Manufacturing and Motion: Realizing AI-powered innovation with MongoDB Atlas

June 27, 2024

Unified Namespace Implementation with MongoDB and MaestroHub

In the complex world of modern manufacturing, a crucial challenge has long persisted: how to seamlessly connect the physical realm of industrial control systems with the digital landscape of enterprise operations. The International Society of Automation's ISA-95 standard, often visualized as the automation pyramid, has emerged as a guiding light. As shown below, this five-level hierarchical model empowers manufacturers to bridge the gap between these worlds, unlocking a path toward smarter, more integrated operations. Figure 1: In the automation pyramid, data moves up or down one layer at a time, using point-to-point connections. Manufacturing organizations face a number of challenges when implementing smart manufacturing applications due to the sheer volume and variety of data generated. An average factory produces terabytes of data daily, including time series data from machines stored in process historians and accessed by supervisory control and data acquisition (or SCADA) systems. Additionally, manufacturing execution systems (MES), enterprise resource planning (ERP) systems, and other operations software generate vast amounts of structured and unstructured data. Globally, the manufacturing industry generates an estimated 1.9 petabytes of data annually . Manufacturing leaders are eager to leverage their data for AI and generative AI projects, but a Workday Global Survey reveals that only 4% of the survey’s respondents believe their data is fully accessible for such applications. Data silos are a significant hurdle, with data workers spending an average of 48% of their time on data search and preparation. A popular approach to making data accessible is consolidating it in a cloud data warehouse and then adding context. However, this can be costly and inefficient, as dumping data without context makes it difficult for AI developers to understand its meaning and origin, especially for operational technology time series data. Figure 2: Pushing uncontextualized data to a data warehouse and then adding context is expensive and inefficient. All these issues underscore the need for a new approach—one that not only standardizes data across disparate shop floor systems, but also seamlessly weaves context into the fabric of this data. This is where the Unified Namespace (UNS) comes in. Figure 3: Unified Namespace provides the right data and context to all the applications connected to it. Unified Namespace is a centralized, real-time repository for all production data. It provides a single, comprehensive view of the business's current state. Using an event-driven architecture, applications publish real-time updates to a central message broker, which subscribers can consume asynchronously. This creates a flexible, decoupled ecosystem where applications can both produce and consume data as needed. Figure 4: UNS enables all the enterprise systems to have one centralized location to get the data they need for what they want to accomplish. MaestroHub and MongoDB: Solving the UNS challenge Initially introduced in 2011 at the Hannover Fair of Industrial Technologies, the core idea behind Industry 4.0 is to establish seamless connectivity and interoperability between disparate systems used in manufacturing. And UNS aims to solve this. Over the past five years, we have seen interest in UNS ramping up steadily, and now manufacturers are looking for practical ways to implement it. In particular, a question we’re frequently asked is where does UNS actually live. To answer that question, we need to look at popular architecture patterns, and the pros and cons of each. The most common pattern is implementing UNS in an MQTT broker. An MQTT broker will act as an intermediary entity that receives messages published by clients, filters the messages by topic, and distributes them to subscribers. The reason most manufacturers choose MQTT is it is an open architecture that is easy to implement. However, the challenge with just using the MQTT broker is that the clients don't get historical data access (which will be required to build the analytical and AI applications). Another approach can be to just dump all the data in a data warehouse and then add context to it. This can solve the problem of historical data access but it is an inefficient approach to standardize messages after they have been landed in the data warehouse in the cloud. A superior solution for comprehensive, real-time data access is combining a single source of truth (SSoT) Unified Namespace platform like MaestroHub with a flexible multi-cloud data platform like MongoDB Atlas. MaestroHub creates SSoT for industrial data, resulting in an up to 80% reduction in integration effort for brownfield facilities. Figure 5: MaestroHub SSoT creates a unified data integration layer, saving up to 50% of time in data contextualization (Source: MaestroHub). MaestroHub provides the connectivity layer to all data sources on the factory floor, along with contextualization and data orchestration. This makes it easy to connect the data needed for the UNS, enrich it with more context, and then publish it to consumers using the protocol that works best for them. Under the hood, MaestroHub stores metadata of connections, instances, and flows, and uses MongoDB as the database to store all this data. MongoDB’s flexible data modeling patterns reduce the complexity of mapping and transforming data when it's shared across different clients in the UNS. Additionally, scalable data indexing overcomes performance concerns as the UNS grows over time. Figure 6: MaestroHub and MongoDB together enable a real-time UNS plus long-term storage. MongoDB: The foundation for intelligent industrial UNS In the quest to build a unified namespace system (UNS) for the modern industrial landscape, the choice of database becomes paramount. So why turn to MongoDB? Scalability and high availability: It scales effortlessly, both vertically and horizontally (sharding), to handle the torrent of data from sensors, machines, and processes. Operational Technology (OT) systems generate vast amounts of data from these sources, and MongoDB ensures seamless management of that information. Document data model: Its adaptable document model accommodates diverse data structures, ensuring a harmonious flow of information. A Unified Namespace (UNS) must handle data from any factory source, accommodating structure variations. MongoDB's flexible schema design allows different data models to coexist in a single database, with schema extensibility at runtime. This flexibility facilitates the seamless integration of new data sources and types into the UNS. Real-time data processing: MongoDB Change Streams and Atlas Device Sync empower third-party applications to access real-time data updates. This is essential for monitoring, alerting, and real-time analysis within a UNS, enabling prompt responses to critical events. Gen AI application development with ease: Atlas Vector Search efficiently performs semantic searches on vector embeddings stored in MongoDB Atlas. This capability seamlessly integrates with large language models (LLMs) to provide relevant context in retrieval-augmented generation (RAG) systems. Given that the Universal Name Service (UNS) functions as a single source of truth for industrial applications, connecting gen AI apps to retrieve context from the UNS database ensures accurate and reliable information retrieval for these applications. With the foundational database established, let's explore MaestroHub, a platform designed to leverage the power of MongoDB in industrial settings. The MaestroHub platform MaestroHub is a provider of a SSoT for industrial data, specifically tailored for manufacturers. It achieves this through: Data connectors: MaestroHub connects to diverse data sources using 38 different industrial communication protocols, encompassing OT drivers, files, databases (SQL, NoSQL, time series), message brokers, web services, cloud systems, historians, and data warehouses. The bi-directional nature of 90% of these protocols ensures comprehensive data integration, leaving no data siloed. Data contextualization based on ISA-95: Leveraging ISA-95 Part 2, MaestroHub employs a semantic hierarchy and a clear naming convention for easy navigation and understanding of data topics. The contextualization of the payload is not just limited to the unit of measure AND definitional but also contains Enterprise/Site/Area/Line/Cell details, which are invaluable for analytics studies. Data contextualization is an important feature of a UNS platform. Logic flows/rule engine: Adhering to the UNS principle "Do not make any assumptions on how the data will be consumed," the data should flow flexibly from sources to brokers and from brokers to consumers in terms of rules, frequencies, and multiple endpoints. MaestroHub allows you to set rules such as Always, OnChange, OnTrue, and WhileTrue, where you can dynamically determine the conditions using events and inputs via JavaScript. Insights created by MaestroHub: MaestroHub provides real-time diagnostics of data health by leveraging Prometheus, Elasticsearch, Fluentd, and Kibana. Network problems, changed endpoints, and changed data types are automatically diagnosed and reported as insights. Additionally, MaestroHub uses NATS for queue management and stream analytics, buffering data in the event of a network outage. This allows IT and OT teams to monitor, debug, and audit logs with full data lineage. Conclusion The ISA-95 automation pyramid presents significant challenges for the manufacturing industry, including a lack of flexibility, limited scalability, and difficulty integrating new technologies. By adopting a Unified Namespace architecture with MaestroHub and MongoDB, manufacturers can overcome these challenges and achieve real-time visibility and control over their operations, leading to increased efficiency and improved business outcomes. Read more on how MongoDB enables Unified Namespace via its multi-cloud modern database. We are actively working with our clients on solving Unified Namespace challenges. Take a look at our Manufacturing and Industrial IoT page for more stories or contact us through the web form in the link.

June 18, 2024

Transforming Predictive Maintenance with AI: Real-Time Audio-Based Diagnostics with Atlas Vector Search

Wind turbines are a critical component in the shift away from fossil fuels toward more sustainable, green sources of energy. According to the International Energy Agency (IEA), the global capacity of wind energy has been growing rapidly, reaching over 743 gigawatts by 2023. Wind energy, in particular, has one of the greatest potentials to increase countries' renewable capacity growth. Solar PV and wind additions are forecast to more than double by 2028 compared with 2022, continuously breaking records over the forecast period. This growth highlights the increasing reliance on wind power and, consequently, the need for effective maintenance strategies. Keeping wind turbines operating at maximum capacity is essential to ensuring their continued contribution to the energy grid. Like any mechanical device, wind turbines must undergo periodic maintenance to keep them operating at optimal levels. In recent years, advancements in technology—particularly in AI and machine learning—have played a significant role by introducing predictive maintenance breakthroughs to industrial processes like periodic maintenance. By integrating AI into renewable energy systems, organizations of all sizes can reduce costs and gain efficiencies. In this post, we will dig into an AI application use case for real-time anomaly detection through sound input, showcasing the impact of AI and MongoDB Atlas Vector Search for predictive maintenance of wind turbines. Check out our AI Learning Hub to learn more about building AI-powered apps with MongoDB. Predictive Maintenance in Modern Industries Companies increasingly invest in predictive maintenance to optimize their operations and drive efficiency. Research from Deloitte indicates that predictive maintenance can reduce equipment downtime by 5–15 percent, increase labor productivity by 5–20 percent, and reduce overall new equipment costs by 3–5 percent. This helps organizations maximize their investment in equipment and infrastructure. By implementing predictive maintenance strategies, companies can anticipate equipment failures before they occur, ultimately resulting in longer equipment lifetimes, tighter budget control, and higher overall throughput. More concretely, businesses aim to reduce mean time to repair, optimal ordering of replacement parts, efficient people management, and reduced overall maintenance costs. Leveraging data interoperability, real-time analysis, modeling and simulation, and machine learning techniques, predictive maintenance enables companies to thrive in today's competitive landscape. However, despite its immense potential, predictive maintenance also presents significant challenges. One major hurdle is the consolidation of heterogeneous data, as predictive maintenance systems often need to integrate data from various formats and sources that can be difficult to integrate. Scalability also becomes a concern when dealing with the high volumes of IoT signals generated by numerous devices and sensors. And lastly, managing and analyzing this vast amount of data in real-time poses challenges that must be overcome to realize the full benefits of predictive maintenance initiatives. At its core, predictive maintenance begins with real-time diagnostics, enabling proactive identification and mitigation of potential equipment failures in real-time. Figure 1: Predictive Maintenance starts with real-time diagnostics However, while AI has been employed for real-time diagnostics for some time, the main challenge has been acquiring and utilizing the necessary data for training AI models. Traditional methods have struggled with incorporating unstructured data into these models effectively. Enter gen AI and vector search technologies, positioned to revolutionize this landscape. Flexible data platforms working together with AI algorithms can help generate insights from diverse data types, including images, video, audio, geospatial data, and more, paving the way for more robust and efficient maintenance strategies. In this context, MongoDB Atlas Vector Search stands out as a foundational element for effective and efficient gen AI-powered predictive maintenance models. Why MongoDB and Atlas Vector Search? For several reasons, MongoDB stands out as the preferred database solution for modern applications. Figure 2: MongoDB Atlas modern database Document data model One of the reasons why the document model is well-suited to the needs of modern applications is its ability to store diverse data types in BSON (Binary JSON) format, ranging from structured to unstructured. This flexibility essentially eliminates the middle layer necessary to convert to a SQL-like format, resulting in easier-to-maintain applications, lower development times, and faster response to changes. Time series collections MongoDB excels in handling time-series data generated by edge devices, IoT sensors, PLCs, SCADA systems, and more. With dedicated time-series collections, MongoDB provides efficient storage and retrieval of time-stamped data, enabling real-time monitoring and analysis. Real-time data processing and aggregation MongoDB's adeptness in real-time data processing is crucial for immediate diagnostics and responses, ensuring timely interventions to prevent costly repairs and downtime. Its powerful aggregation capabilities facilitate the synthesis of data from multiple sources, providing comprehensive insights into fleet-wide performance trends. Modern database Beyond just storing data, MongoDB Atlas is a multi-cloud developer data platform, providing the flexibility required to build a diverse range of applications. Atlas includes features like transactional processing, text-based search, vector search, in-app analytics, and more through an elegant and integrated suite of data services. It offers developers a top-tier experience through a unified query interface, all while meeting the most demanding requirements for resilience, scalability, and cybersecurity. Atlas Vector Search Among the out-of-the-box features offered by MongoDB Atlas, Atlas Vector Search stands out, enabling the search of unstructured data effortlessly. You can generate vector embeddings with machine learning models like the ones found in OpenAI or Hugging Face, and store and index them in Atlas. This feature facilitates the indexing of vector representations of objects and retrieves those that are semantically most similar to your query. Explore the capabilities of Atlas Vector Search . This functionality is especially interesting for unstructured data that was previously hard to leverage, such as text, images, and audio, allowing searches that combine audio, video, metadata, production equipment data, or sensor measurements to provide an answer to a query. Let's delve into how simple it is to leverage AI to significantly enhance the sophistication of predictive maintenance models with MongoDB Atlas. Real-time audio-based diagnostics with Atlas Vector Search In our demonstration, we'll showcase real-time audio-based diagnostics applied to a wind turbine. It's important to note that while we focus on wind turbines here, the concept can be extrapolated to any machine, vehicle, or device emitting sound. To illustrate this concept, we'll utilize a handheld fan as our makeshift wind turbine. Wind turbines emit different sounds depending on their operational status. By continuously monitoring the turbine’s audio, our system can accurately specify the current operational status of the equipment and reduce the risk of unexpected breakdowns. Early detection of potential issues allows for enhanced operational efficiency, minimizing the time and resources required for manual inspections. Additionally, timely identification can prevent costly repairs and reduce overall turbine downtime, thus enhancing cost-effectiveness. Now, let’s have a look at how this demo works! Figure 3: Application Architecture Audio Preparation We begin by capturing the audio from the equipment in different situations (normal operation, high vs. low load, equipment obstructed, not operating, etc.). Once each sound is collected, we use an embedding model to process the audio data to convert it to a vector. This step is crucial because by generating embeddings for each audio track, which are high-dimensional vector representations, we are essentially capturing the unique characteristics of the sound. We then upload these vector embeddings to MongoDB Atlas. By adding just a few examples of sounds in our database, they are ready to be searched (and essentially compared) with the sound emitted by our equipment during its operation in real-time. Audio-based diagnosis Now, we put our equipment into normal operation and start capturing the sound it is making in real-time. In this demonstration, we capture one-second clips of audio. Then, with the same embedding model used before, we take our audio clips and convert them to vector embeddings in real-time. This process happens in milliseconds, allowing us to have real-time monitoring of the audio. The one-second audio clips, now converted to vector embeddings, are then sent to MongoDB Atlas Vector Search, which can search for and find the most similar vectors from the ones we previously recorded in our audio preparation phase. The result is given back with a percentage of similarity, enabling a very accurate prediction of the current status of the operation of the wind turbine. These steps are performed repeatedly every second, leveraging fast embedding of vectors and quick searches, allowing for real-time monitoring based on sound. Check out the video below to see it in action! Transforming Predictive Maintenance with AI and MongoDB Predictive maintenance offers substantial benefits but poses challenges like data integration and scalability. MongoDB stands out as a preferred database solution, offering scalability, flexibility, and real-time data processing. As technology advances, AI integration promises to further revolutionize the industry. Thank you to Ralph Johnson and Han Heloir for their valuable contributions to this demo! Head over to our quick-start guide to get started with Atlas Vector Search today. Explore how MongoDB empowers manufacturing operations by visiting these resources: Generative AI in Predictive Maintenance Applications Transforming Industries with MongoDB and AI: Manufacturing and Motion MongoDB for Automotive: Driving Innovation from Factory to Finish Line

May 28, 2024

Transforming Industries with MongoDB and AI: Manufacturing and Motion

This is the first in a six-part series focusing on critical AI use cases across several industries . The series covers the manufacturing and motion, financial services, retail, telecommunications and media, insurance, and healthcare industries. The integration of artificial intelligence (AI) within the manufacturing and automotive industries has transformed the conventional value chain, presenting a spectrum of opportunities. Leveraging Industrial IoT, companies now collect extensive data from assets, paving the way for analytical insights and unlocking novel AI use cases, including enhanced inventory management and predictive maintenance. Inventory management Efficient supply chains can control operational costs and ensure on-time delivery to their customers. Inventory optimization and management is a key component in achieving these goals. Managing and optimizing inventory levels, planning for fluctuations in demand, and of course, cutting costs are all imperative goals. However, efficient inventory management for manufacturers presents complex data challenges too, primarily in forecasting demand accurately and optimizing stock levels. This is where AI can help. Figure 1: Gen AI-enabled demand forecasting with MongoDB Atlas AI algorithms can be used to analyze complex datasets to predict future demand for products or parts. Improvement in demand forecasting accuracy is crucial for maintaining optimal inventory levels. AI-based time series forecasting can assist in adapting to rapid changes in customer demand. Once the demand is known, AI can play a pivotal role in stock optimization. By analyzing historical sales data and market trends, manufacturers can determine the most efficient stock levels and even reduce human error. On top of all this existing potential, generative AI can help with generating synthetic inventory data and seasonally adjusted demand patterns. It can also help with creating scenarios to simulate supply chain disruptions. MongoDB Atlas makes this process simple. At the warehouse, the inventory can be scanned using a mobile device. This data is persisted in Atlas Device SDK and synced with Atlas using Device Sync, which is used by MongoDB customers like Grainger . Atlas Device Sync provides an offline-first seamless mobile experience for inventory tracking, making sure that inventory data is always accurate in Atlas. Once data is in Atlas, it can serve as the central repository for all inventory-related data. This repository becomes the source of data for inventory management AI applications, eliminating data silos and improving visibility into overall inventory levels and movements. Using Atlas Vector Search and generative AI, manufacturers can easily categorize products based on their seasonal attributes, cluster products with similar seasonal demand patterns, and provide context to the foundation model to improve the accuracy of synthetic inventory data generation. Predictive maintenance The most basic approach to maintenance today is reactive — assets are deliberately allowed to operate until failures actually occur. The assets are maintained as needed, making it challenging to anticipate repairs. Preventive maintenance, however, allows systems or components to be replaced based on a conservative schedule to prevent commonly occurring failures — although predictive maintenance is expensive to implement due to frequent replacement of parts before end-of-life. Figure 2: Audio-based anomaly detection with MongoDB Atlas. Scan the QR code to try it out yourself. AI offers a chance to efficiently implement predictive maintenance using data collected from IoT sensors on machinery trained to detect anomalies. ML/AI algorithms like regression models or decision trees are trained on the preprocessed data, deployed on-site for inference, and continuously analyzed sensor data. When anomalies are detected, alerts are generated to notify maintenance personnel, enabling proactive planning and execution of maintenance actions to minimize downtime and optimize equipment reliability and performance. A retrieval-augmented generation (RAG) architecture can be deployed to generate or curate the data preprocessor removing the need for specialized data science knowledge. The domain expert can provide the right prompts for the large language model. Once the maintenance alert is generated by an AI model, generative AI can come in again to suggest a repair strategy, taking spare parts inventory data, maintenance budget, and personal availability into consideration. Finally, the repair manuals can be vectorized and used to power a chatbot application that guides the technician in performing the actual repair. MongoDB documents are inherently flexible while allowing data governance when required. Since machine health prediction models require not just sensor data but also maintenance history and inventory data, the document model is a perfect fit to model such disparate data sources. During the maintenance and support process of a physical product, information such as product information and replacement parts documentation must be available and easily accessible to support staff. Full-text search capabilities provided by Atlas Search can be integrated with the support portal and help staff retrieve information from Atlas clusters with ease. Atlas Vector Search is a foundational element for effective and efficiently powered predictive maintenance models. Manufacturers can use MongoDB Atlas to explore ways of simplifying machine diagnostics. Audio files can be recorded from machines, which can then be vectorized and searched to retrieve similar cases. Once the cause is identified, they can use RAG to implement a chatbot interface that the technician can interact with and get context-aware, step-by-step guidance on how to perform the repair. Autonomous driving With the rise of connected vehicles, automotive manufacturers have been compelled to transform their business models into software-first organizations. The data generated by connected vehicles is used to create better driver assistance systems, paving the way for autonomous driving applications. However, it is challenging to create fully autonomous vehicles that can drive safer than humans. Some experts estimate that the technology to achieve level 5 autonomy is about 80% developed — but the remaining 20% will be extremely hard to achieve and will take a lot of time to perfect. Figure 3: MongoDB Atlas’s Role in Autonomous Driving AI-based image and object recognition in automotive applications face uncertainties, but manufacturers must utilize data from radar, LiDAR, cameras, and vehicle telemetry to improve AI model training. Modern vehicles act as data powerhouses, constantly gathering and processing information from onboard sensors and cameras, generating significant Big Data. Robust storage and analysis capabilities are essential to manage this data, while real-time analysis is crucial for making instantaneous decisions to ensure safe navigation. MongoDB can play a significant role in addressing these challenges. The document model is an excellent way to accommodate diverse data types such as sensor readings, telematics, maps, and model results. New fields to the documents can be added at run time, enabling the developers to easily add context to the raw telemetry data. MongoDB’s ability to handle large volumes of unstructured data makes it suitable for the constant influx of vehicle-generated information. Atlas Search provides a performant search engine to allow data scientists to iterate their perception AI models. Finally, Atlas Device Sync can be used to send configuration updates to the vehicle's advanced driving assistance system Other notable use cases AI plays a critical role in fulfilling the promise of Industry 4.0. Numerous other use cases of AI can be enabled by MongoDB Atlas, some of which include: Logistics Optimization: AI can help optimize routes resulting in reduced delays and enhanced efficiency in day-to-day delivery operations. Quality Control and Defect Detection: Computer or machine vision can be used to identify irregularities in the products as they are manufactured. This ensures that product standards are met with precision. Production Optimization: By analyzing time series data from sensors installed on production lines, waste can be identified and reduced, thereby improving throughput and efficiency. Smart After Sales Support: Manufacturers can utilize AI-driven chatbots and predictive analytics to offer proactive maintenance, troubleshooting, and personalized assistance to customers. Personalized Product Recommendations: AI can be used to analyze user behavior and preferences to deliver personalized product recommendations via a mobile or web app, enhancing customer satisfaction and driving sales. The integration of AI in manufacturing and automotive industries has revolutionized traditional processes, offering a plethora of opportunities for efficiency and innovation. With industrial IoT and advanced analytics, companies can now harness vast amounts of data to enhance inventory management and predictive maintenance. AI-driven demand forecasting ensures optimal stock levels, while predictive maintenance techniques minimize downtime and optimize equipment performance. Moreover, as automotive manufacturers work toward autonomous driving, AI-powered image recognition and real-time data analysis become paramount. MongoDB Atlas emerges as a pivotal solution, providing flexible document modeling and robust storage capabilities to handle the complexities of Industry 4.0. Beyond the manufacturing and automotive sectors, the potential of AI-enabled by MongoDB Atlas extends to logistics optimization, quality control, production efficiency, smart after-sales support, and personalized customer experiences, shaping the future of Industry 4.0 and beyond. Learn more about AI use cases for top industries in our new white paper, “ How Leading Industries are Transforming with AI and MongoDB Atlas .” Head over to our quick-start guide to get started with Atlas Vector Search today.

March 19, 2024

Integrate OPC UA With MongoDB - A Feasibility Study With Codelitt

Open Platform Communications Unified Architecture (OPC UA) is a widely recognized and important communication standard for Industry 4.0 and industrial IoT. It enables interoperability across different machines and equipment, ensuring reliable and secure information sharing within the Operational Technology (OT) layer. By providing a standard framework for communication, OPC UA enhances data integrity, security, and accessibility of data, enabling many use cases for Industry 4.0. OPC UA focuses on standard data transmission and information modeling. It uses multiple data encoding methods, such as binary or JavaScript Object Notation (JSON), and leverages different levels of security encryption to address security concerns. For information modeling, it adopts an object-oriented approach to abstract and model specific industrial assets such as robots, machines, and processes. Rich data models and object types can be created for a description of machine attributes and composition. Using OPC UA, the traditional context-less time-series machine data is transformed into a semantic-based information model. MongoDB's document model offers a straightforward and compelling approach for storing OPC UA semantic information models due to its flexibility and compatibility with complex data structures. The document model is a superset of all other types of data models, which makes it very popular in the developer community. OPC UA information models contain detailed relationships and hierarchies, making the dynamic schema of MongoDB a natural fit. Fields in the document are extensible at run time, making dynamic updates and efficient querying a breeze. For example, consider an OPC UA information model representing an industrial robot. This model will encompass information about the robot's status, current task, operational parameters, and maintenance history. Example OPC UA information model for an Industrial Robot Robot RobotName (Variable) Status (Variable) CurrentTask (Variable) OperationalParameters (Object) MaxSpeed (Variable) PayloadCapacity (Variable) Reach (Variable) MaintenanceHistory (Array of Objects) Timestamp (Variable) Description (Variable) With MongoDB, this model can be easily represented in a document with nested fields. { "_id": ObjectId("654321ab12345abcd6789"), "RobotName": "Robot1", "Status": "Running", "CurrentTask": "Assembling Component ABC", "OperationalParameters": { "MaxSpeed": 80, // in cm/s "PayloadCapacity": 150, // in kg "Reach": 2.65 // in m }, "MaintenanceHistory": [ { "Timestamp": "2023-08-25T10:00:00", "Description": "Routine checkup" }, { "Timestamp": "2023-06-25T14:30:00", "Description": "Replaced worn-out gripper" } ] } This MongoDB document easily captures the complexities of the OPC UA information model. Hierarchical attributes in the model are maintained as objects, and arrays can represent historical data and log files. As the robot runs during the production shift, the document can be easily updated with real-time status information. Instead of worrying about creating a complicated Entity Relationship diagram with SQL databases, MongoDB offers a superior alternative to represent digital shadows of industrial equipment. Now that we have seen how easy it is to model OPC UA data in MongoDB, let's talk about how to connect an OPC UA server to MongoDB. One of our partners, Codelitt, is developing a connector that can ingest time-series OPC UA data into MongoDB in real time. Codelitt is a custom software strategy, engineering, and design company. The architecture of the end-to-end solution is shown in Figure 1. Figure 1: High-level architecture and data flow In Figure 1: Industrial equipment and controllers will transmit data to local servers using the OPC UA protocol. OPC UA servers will listen to these devices and broadcast them to all subscribed clients. Clients will listen to specific events/variables and queue the event to be stored. The message broker will provide the queue system to digest a large amount of data and provide reliability between the event source and the data storage. MongoDB Atlas will provide the final destination of data, and the ability to do analytics using the aggregation framework and visualization using Atlas Charts. Technical details It is assumed that the user already has machines that have OPC UA server enabled. For the OPC UA client, depending on the client's preferences, the Codelitt solution can switch between a custom-built OPC UA client based on the Node-OPCUA open source project, AWS IoT SiteWise Edge , or a Confluent-based OPC UA source connector . In the case of a custom-built client, it will connect to the machine's OPC UA server using OPC TCP and extract the necessary data that is then transmitted to a broker. The message broker could be any cloud-provided solution (Azure Event Hub, Amazon Kinesis, etc.) or any form of Kafka implementation from Confluence for example. In the case of Kafka, the MongoDB Kafka connector can be leveraged to push data to the database. Finally, leveraging the aggregation framework , the operating parameters of each device are queried for visualization via MongoDB Atlas Charts . In summary, the MongoDB document model elegantly mirrors OPC UA information, and there are multiple options available to users who would like to push data from their OPC UA servers to MongoDB. To learn more about MongoDB’s role in the manufacturing sector, please visit our manufacturing webpage . To learn more about how Codelitt is digitally transforming industries, please visit their website .

January 16, 2024

How MongoDB and Alibaba Cloud are Powering the Era of Autonomous Driving

The emergence of autonomous driving technologies is transforming how automotive manufacturers operate, with data taking center stage in this transformation. Manufacturers are now not only creators of physical products but also stewards of vast amounts of product and customer data. As vehicles transform into connected vehicles, automotive manufacturers are compelled to transform their business models into software-first organizations. The data generated by connected vehicles is used to create better driver assistance systems and paves the way for autonomous driving applications. It has to be noted that the journey toward autonomous vehicles is not just about building reliable vehicles but harnessing the power of connected vehicle data to create a new era of mobility that seamlessly integrates cutting-edge software with vehicle hardware. The ultimate goal of autonomous vehicle makers is to produce cars that are safer than human-driven vehicles. Since 2010, investors have poured over 200 billion dollars into autonomous vehicle technology. Even with this large amount of investment, it is very challenging to create fully autonomous vehicles that can drive safer than humans. Some experts estimate that the technology to achieve level 5 autonomy is about 80% developed but the last 20% will be extremely hard to achieve and will take a lot of time to perfect. Unusual events such as extreme weather, wildlife crossings, and highway construction are still enigmas for many automotive companies to solve. Check out our AI resource page to learn more about building AI-powered apps with MongoDB. The answer to these challenges is not straightforward. AI-based image and object recognition still has a long way to go to deal with uncertainties on the road. However, one thing is certain, automotive manufacturers need to make use of data captured by radar, LiDAR, camera systems, and the whole telemetry system in the vehicle in order to train their AI models better. A modern vehicle is a data powerhouse. It constantly gathers and processes information from onboard sensors and cameras. The Big Data generated as a result presents a formidable challenge, requiring robust storage and analysis capabilities. Additionally, this time series data needs to be analyzed in real-time and decisions have to be made instantaneously in order to guarantee safe navigation. Furthermore, ensuring data privacy and security is also a hurdle to cross since self-driving vehicles need to be shielded from cyber attacks as such an attack can cause life-threatening events. The development of high-definition (HD) maps to help the vehicle ‘see’ what is on the road also poses technical challenges. Such maps are developed using a combination of different data sources such as Global Navigation Satellite Systems (GNSS), radar, IMUs, cameras, and LiDAR. Any error in any one of these systems aggregates and ultimately impacts the accuracy of the navigation. It is required to have a data platform in the middle of the data source (vehicle systems) and the AI platform to accommodate and consolidate this diverse information while keeping this data secure. The data platform should be able to preprocess this data as well as add additional context to it before using it to train or run the AI modules such as object detection, semantic segmentation, and path planning. MongoDB can play a significant role in addressing above mentioned data-related challenges posed by autonomous driving. The document model is an excellent way to accommodate diverse data types such as sensor readings, telematics, maps, and model results. New fields to the documents can be added at run time, enabling the developers to easily add context to the raw telemetry data. MongoDB’s ability to handle large volumes of unstructured data makes it suitable for the constant influx of vehicle-generated information. MongoDB is not only an excellent choice for data storage but also provides comprehensive data pre-processing capabilities through its aggregation framework. Its support for time series window functions allows data scientists to produce calculations over a sorted set of documents. Time series collections also dramatically reduce storage costs. Column compression significantly improves practical compression, reduces the data's overall storage on disk, and improves read performance. MongoDB offers robust security features such as role-based access control, encryption at rest and in transit, comprehensive auditing, field-level redaction and encryption, and down to the level of client-side field-level encryption that can help shield sensitive data from potential cyber threats while ensuring compliance with data protection regulations. For challenges related to effectively storing and querying HD maps, MongoDB’s geospatial features can aid in querying location-based data and also combining the information from maps with telemetry data fulfilling the continuous updates and accuracy requirements for mapping. Furthermore, MongoDB's horizontal scaling or sharding allows for the seamless expansion of storage and processing capabilities as the volume of data grows. This scalability is essential for handling the data streams generated by fleets of self-driving vehicles. During the research and development of autonomous driving projects, scalable infrastructure is required to quickly and steadily collect and process massive data. In such projects, data is generated at the terabyte level every day. To meet these needs, Alibaba Cloud provides a solution that integrates data collection, transmission, storage, and computing. In this solution, the data collected daily by sensors can be simulated and collected using Alibaba Cloud Lightning Cube and sent to the Object Storage Service (OSS) . Context is added to this data using a translator and then this contextualized information can be pushed to MongoDB to train models. MongoDB and Alibaba Cloud recently announced a four-year extension to their strategic global partnership that has seen significant growth since being announced in 2019. Through this partnership, automotive manufacturers can easily set up and use MongoDB-as-a-service-ApsaraDB for MongoDB from Alibaba Cloud’s data centers globally. Figure 1: Data collection and model training data link with MongoDB on Alibaba Cloud. When the vehicle is on the road, the telemetry data is captured through an MQTT gateway, converted into Kafka, and then pushed into MongoDB for data storage and archiving. This data can be used for various applications such as real-time status updates for engine and battery, accident analysis, and regulatory reporting. Figure 2: Mass Production vehicles data link with MongoDB on Alibaba Cloud For a company that is looking to build autonomous driving assistance systems, Alibaba Cloud and ApsaraDB for MongoDB is an excellent technology partner to have. ApsaraDB for MongoDB can handle TBs of diverse sensor data from cars on a daily basis, which doesn't conform to a fixed format. MongoDB provides reliable and highly available data storage for this heterogenous data enabling companies to rapidly expand their system within minutes resulting in time savings when processing and integrating autonomous driving data. By leveraging Alibaba Cloud's ApsaraDB for MongoDB, the R&D team can focus on innovation rather than worrying about data storage and scalability, contributing to faster innovation in the field of autonomous driving. In summary, MongoDB's flexibility, versatility, scalability, real-time capabilities, and strong security framework make it well-suited to address the multifaceted data requirements and challenges that autonomous driving presents. By efficiently managing and analyzing the Big Data generated, MongoDB and Alibaba Cloud are paving the path toward reliable and safe self-driving technology. To learn more about MongoDB’s role in the automotive industry, please visit our manufacturing and automotive webpage .

September 11, 2023

Empowering Automotive Developers for the Road Ahead

MongoDB 7.0 is here, and companies across industries are benefiting from being early adopters of cutting-edge data platform technology. Let’s take a closer look at the automotive industry specifically, and how many of MongoDB’s new features and capabilities can revolutionize the way automotive developers build, iterate, and scale their applications. In the fast-changing automotive landscape, development teams face the challenge of delivering compelling user experiences faster and smarter than ever before. MongoDB's modern database becomes a vital tool for developers striving to innovate quickly and efficiently, supporting a wide range of application use cases while streamlining development and ensuring optimal performance. MongoDB Atlas Stream Processing MongoDB Atlas Stream Processing , coming soon in private preview, will be a game-changing advantage for the automotive industry, offering real-time data insights and rapid responses to critical events. As vehicles generate an ever-increasing stream of sensor data, this capability enables automotive developers to process, analyze, and act upon data in real-time. Manufacturers and fleet management companies can monitor vehicle health, track performance, and optimize maintenance schedules on the fly, while proactive safety measures and anomaly detection ensure utmost safety for drivers and passengers. Moreover, MongoDB Atlas Stream Processing enables developers to unlock the potential of connected car applications, making real-time data processing imperative for intelligent navigation, personalized infotainment services, and efficient route planning . MongoDB Atlas Vector Search MongoDB Atlas Vector Search , currently in public preview, holds immense potential for revolutionizing the automotive industry. By utilizing vector representations of unstructured data such as audio, images, and text, MongoDB Atlas enables developers to store, index, and query data based on similarities in high-dimensional vector spaces alongside operational data. For the automotive industry, this means unlocking a world of possibilities in data analysis, anomaly detection, and predictive maintenance. In fact, as mentioned in the MongoDB.local Chicago keynote , a top 10 auto manufacturer leveraged Vector Search to enable engine diagnostics based on engine audio. Watch the video below to learn more. Atlas Vector Search empowers automotive developers to create smarter, data-driven applications that deliver more relevant and accurate insights, ultimately enhancing the driving experience and safety for all. MongoDB Atlas Vector Search allows manufacturers to query and qualify possible equipment and product failure causes and get AI-generated recommendations on how to adjust operational parameters and extend the life of their equipment and products. The automotive industry thrives on innovation and efficiency, and Atlas Vector Search opens new avenues for optimizing vehicle performance, predicting maintenance needs, and enhancing overall user experiences on the road. MongoDB Relational Migrator In the ever-evolving automotive industry, legacy relational databases often pose challenges in scalability, flexibility, and performance. Relational databases are very prevalent in the manufacturing industry and hinder innovation due to rigid data models and limited scalability. MongoDB Relational Migrator addresses these pain points by assisting with several critical steps in the path to modernization for automotive developers. By migrating data from common relational databases to MongoDB, automotive companies can break free from the limitations of legacy systems and embrace the full potential of a NoSQL database . This migration process streamlines data transfer, offers valuable data modeling recommendations, and empowers developers to refactor applications quickly and efficiently. Embracing MongoDB's flexible document data model optimizes performance, scales applications effortlessly, and unlocks the potential for real-time analytics, enabling the industry to stay ahead in the race for innovation. MongoDB Relational Migrator becomes a catalyst for driving transformative change in the automotive sector, enabling faster and more efficient data processing for mission-critical applications and paving the way for sophisticated AI-driven solutions. As automotive companies embrace data-driven insights and strive to deliver unparalleled user experiences, MongoDB Relational Migrator empowers the industry to leverage the full potential of NoSQL databases, enabling automotive applications to zoom ahead in the fast lane of innovation. MongoDB 7.0 promises to be a game-changer for developers across industries , empowering them to build innovative, scalable, and secure applications that drive the future. With the power of MongoDB, developers can accelerate their journey toward automotive innovation and build the vehicles and experiences of tomorrow. Watch the full MongoDB.local lineup to learn more .

August 22, 2023

Real-Time Energy Monitoring for Smart Buildings with MongoDB and HiveMQ

The Internet of Things (IoT) has ushered in a new era of energy efficiency, enabling the deployment of energy-efficient sensors for energy conservation and resource utilization. With over 1.5 billion connected IoT devices already installed in commercial smart buildings in 2022 and a projected surge to 3.25 billion devices by 2028, the volume of data generated is staggering. To put it in perspective, an average home in 2020 would generate approximately 4.7 terabytes of data annually. However, managing and harnessing this immense amount of real-time streaming data poses a significant challenge for developers. In smart buildings, where a multitude of IoT sensors continuously gather event streaming data, developers often grapple with integrating disparate technologies and investing significant time into data streaming integration. In this blog, we present a simple yet powerful solution to this challenge. We will demonstrate how you can effortlessly move IoT data using standard protocols such as MQTT to MongoDB Atlas using the HiveMQ Enterprise Extension for MongoDB . By optimizing smart buildings with real-time energy monitoring through the seamless integration of MongoDB and HiveMQ, we unlock the potential for efficient energy management and a sustainable future. Let’s get started! Dream team: HiveMQ + MongoDB In a world where energy conservation and efficient resource utilization are essential, let’s go through Figure 1 and see how simple it is to use MongoDB, HiveMQ’s MQTT broker, and the Enterprise Extension for MongoDB to enable real-time energy monitoring for smart building. Figure 1: Combining HiveMQ and MongoDB process data in real-time Step 1: Data transmission Using MQTT-based IoT devices deployed throughout the building, electricity consumption, temperature, and occupancy data is collected and sent to the HiveMQ MQTT broker. The MQTT broker acts as a central hub, efficiently and securely handling the communication between devices and backend systems. The HiveMQ MQTT broker also ensures reliable message delivery. It also provides MQTT-specific features like quality of service, session management, and topic-based message routing. Step 2: Data ingestion The HiveMQ MongoDB extension seamlessly integrates with MongoDB, allowing for persistent storage of the MQTT data in a highly scalable and flexible manner. The fully customizable templating system allows MQTT data to be stored according to the building’s specific operational requirements. MongoDB's document-based model accommodates the varying data formats and structures generated by different IoT devices. Step 3: Data visualization and analytics Once the MQTT data is securely stored in MongoDB, using its powerful in-app analytics, building managers can gain deep insights into energy consumption patterns, identify anomalies, and optimize energy usage. By leveraging MongoDB's rich query support and aggregation framework, building managers can make data-driven decisions promptly, reducing costs and enhancing sustainability. In cases where data needs to be exported to an ML/AI engine, MongoDB Spark and Kafka connectors can be used. Users of MongoDB Atlas can leverage Atlas Device Sync and Realm to send real-time alerts and messages to mobile devices. Data can be visualized using MongoDB Atlas Charts or through a third-party Business Intelligence (BI) tool connected via MongoDB BI connector or Atlas SQL interface. Conclusion By seamlessly integrating HiveMQ's MQTT broker with MongoDB, developers can efficiently handle data transmission, ingestion, and storage. This integration enables building managers to gain valuable insights into energy consumption patterns, make data-driven decisions, and optimize energy usage. To learn more about MongoDB’s role in IoT, please visit our IoT webpage . You can also try the HiveMQ platform now with the Enterprise Extension for MongoDB for free . Thank you Ainhoa Múgica for her contributions to this blog.

July 11, 2023