Applications
Customer stories, use cases, and experiences of MongoDB
Mobile and Edge Solutions with MongoDB and Ditto
September 3, 2024
Applied
THL Simplifies Architecture with MongoDB Atlas Search
Tourism Holdings Limited (THL) originally became a MongoDB customer in 2019, using MongoDB Atlas to help manage a wide variety of telematics data. I was very excited to welcome Charbel Abdo, Solutions Architect for THL at MongoDB .local Sydney in July 2024 to hear more about how the company has significantly expanded its use of MongoDB. The largest RV rental company in the world, THL has branches in New Zealand (where it is headquartered), Australia, the US, Canada, the UK and Europe. Specializing in building, renting, and selling camper vans, THL has a number of well-known brands under its umbrella. In recent years, THL has made a number of significant digital transformation and technology stack optimization efforts, moving from a ‘bolt-on’ approach that necessitated the use of a distributed search and analytics engine to an integrated search solution with MongoDB Atlas . THL operates a complex ecosystem managed by their in-house platform, Motek, which handles booking, pricing, fleet management, and more—with MongoDB Atlas as the central database. Its +7,000 RVs are fitted with telematics devices that send information—such as location, high-speed events, engine problems, and geofences or restricted areas (for example, during the Australian bushfires of 2020)—to vehicles’ onboard computers. THL initially used a bolt-on approach for complex search functionalities by extending their deployment footprint to include a stand-alone instance of Elasticsearch. This setup, while functional, introduced significant data synchronization and performance issues, as well as increased maintenance overhead. Elasticsearch struggled under heavy loads which led to critical failures and system instability, resulting in THL experiencing frequent outages and data inconsistencies. After two years of coping with these challenges, THL resolved to migrate away from ElasticSearch. After doing due diligence, they identified the MongoDB developer data platform’s integrated Search capabilities as the optimum solution. "A couple of months later, we had migrated everything," said Abdo. "Kudos to the MongoDB account team. They were exceptional." The migration process turned out to be relatively straightforward. By iteratively replacing Elasticsearch with MongoDB Atlas Search , THL was able to simplify its architecture, reduce costs, and eliminate the synchronization issues that had plagued the system. The simplification also led to significant performance and reliability improvements. Because it no longer needed the dedicated sync resources processing millions upon millions of records per day, THL was able to turn off its Elasticsearch cluster and to consolidate its resources. “All data sync related issues were gone, eliminated. But also we got our Friday afternoons back, which is always a good thing!” added Abdo. Abdo’s team can now also use existing monitoring tools rather than having to set up something completely separate from the standalone search engine they were using. “Sometimes, changes are easier than you think,” said Abdo. “We spent two-and-a-half years with our faulty solutions just looking for ways to patch up all the problems that we were having. We tried everything except actually looking into how much it would actually take to migrate. We wasted so much time, so much effort, so much money. While if we had thought about this a couple of years ago, it would have been a breeze.” “Over-engineering is bad, simple is better,” he noted. To learn more about how MongoDB Atlas Search can help you build or deepen your search capabilities, visit our MongoDB Atlas Search page .
Ahamove Rides Vietnam’s E-commerce Boom with AI on MongoDB
The energy in Vietnam’s cities is frenetic as millions of people navigate the busy streets with determination and purpose. Much of this traffic is driven by e-commerce, with food and parcel deliveries perched on the back of the country’s countless motorcycles or in cars and trucks. In the first quarter of 2024, online spending in Vietnam grew a staggering 79% over the previous year. Explosive growth like this is expected to continue, raising the industry’s value to $32 billion by 2025 , with 70% of the country’s 100 million population making e-commerce transactions . With massive numbers like this, in logistics, efficiency is king. The high customer expectations for rapid deliveries drive companies like Ahamove to innovate their way to seamless operations with cloud technology. Ahamove is Vietnam’s largest on-demand delivery company, handling more than 200,000 e-commerce, food, and warehouse deliveries daily, with 100,000 drivers and riders plying the streets nationwide. The logistics leader serves a network of more than 300,000 merchants, including regional e-commerce giants like Lazada and Shopee, as well as nationwide supermarket chains and small restaurants. The stakes are high for all involved, so maximizing efficiency is of utmost importance. Innovating to make scale count Online shoppers’ behavior is rarely predictable, and to cope with sudden spikes in daily delivery demand, Ahamove needed to efficiently scale up its operations to enhance customer and end-user satisfaction. Moving to MongoDB Atlas on Amazon Web Services (AWS) in 2019, Ahamove fundamentally changed its ability to meet the rising demand for deliveries and new services that please e-commerce providers, online shoppers, and diners. The scalability of MongoDB is crucial for Ahamove, especially during peak times, like Christmas or Lunar New Year, when the volume of orders surges to more than 200,000 a day. “MongoDB's ability to scale ensures that the database can handle increased loads, including data requests, without compromising performance and leading to quicker order processing and improved user experience,” said Tien Ta, Strategic Planning Manager at Ahamove. One of the powerful services that improves e-commerce across Vietnam is geospatial queries enabled by MongoDB. Using this geospatial data associated with specific locations on Earth's surface, Ahamove can easily locate drivers, map drivers to restaurants to accelerate deliveries, and track orders without relying on third-party services to provide information, which slows deliveries. Meanwhile, the versatility of MongoDB’s developer data platform empowers Ahamove to store its operational data, metadata, and vector embeddings on MongoDB Atlas and seamlessly use Atlas Vector Search to index, retrieve, and build performant generative artificial intelligence (AI) applications. AI evolution Powered by MongoDB Atlas , Ahamove is transforming Vietnam’s e-commerce industry with innovations like instant order matching, real-time GPS vehicle tracking, generative AI chatbots, and services like driver rating and variable delivery times, all available 24 hours a day, seven days a week. In addition to traffic, Vietnam is also famous for its excellent street food. Recognizing the importance of the country’s rapidly growing food and beverage (F&B) industry, which is projected to be worth more than US$27.3 billion in 2024 , Ahamove decided to help Vietnam’s small food vendors benefit from the e-commerce boom gripping the country. Using the latest models, including ChatGPT-4o-mini and Llama 3.1, Ahamove’s fully automated generative AI chatbot on MongoDB integrates with restaurants’ Facebook pages. This makes it easier for hungry consumers to handle the entire order process with the restaurant in natural language, from seeking recommendations to placing orders, making payments, and tracking deliveries to their doorsteps. How AhaFood AI chatbot automates the food order journey “Vietnam’s e-commerce industry is growing rapidly as more people turn to their mobile devices to purchase goods and services,” added Ta. “With MongoDB, we meet this customer need for new purchase experiences with innovative services like generative AI chatbots and faster delivery times.” Anticipated to achieve 10% of food deliveries at Da Nang market and take the solution nationwide in the first half of 2025, AhaFood.AI - Ahamove’s latest initiative, also provides personalized dish recommendations based on consumer demographics, budgets, or historical preferences, helping people find and order their favorite food faster. Moreover, merchants receive timely notifications of incoming orders via the AhaMerchant web portal, allowing them to start preparing dishes earlier. AhaFood.AI also collects and securely stores users’ delivery addresses and phone numbers, ensuring better driver assignment and fulfilling food orders in less than 15 minutes. “Adopting MongoDB Atlas was one of the best decisions we’ve ever made for Ahamove, allowing us to build an effective infrastructure that can scale with growing demand and deliver a better experience for our drivers and customers,” said Ngon Pham, CEO, Ahamove. “Generative AI will significantly disrupt the e-commerce and food industry, and with MongoDB Vector Search we can rapidly build new solutions using the latest database and AI technology.” The vibrant atmosphere of Vietnam's bustling cities is part of the country's charm. Rather than seeking to bring calm to this energy, Vietnam thrives on it. Focusing on improving efficiency and supporting street food vendors in lively urban areas with cloud technology will benefit all. Learn how to build AI applications with MongoDB Atlas . Head over to our quick-start guide to get started with Atlas Vector Search today.
Top Use Cases for Text, Vector, and Hybrid Search
Search is how we discover new things. Whether you’re looking for a pair of new shoes, the latest medical advice, or insights into corporate data, search provides the means to unlock the truth. Search habits—and the accompanying end-user expectations—have evolved along with changes to the search experiences offered by consumer apps like Google and Amazon. The days of the standard of 10 blue links may well be behind us, as new paradigms like vector search and generative AI (gen AI) have upended long-held search norms. But are all forms of search created equal, or should we be seeking out the right “flavor” of search for specific jobs? In this blog post, we will define and dig into various flavors of search, including text, vector and AI-powered search, and hybrid search, and discuss when to use each, including sample use cases where one type of search might be superior to others. Information retrieval revolutionized with text search The concept of text search has been baked into user behavior from the early days of the web, with the rudimentary text box entry and 10 blue link results based on text relevance to the initial query. This behavior and associated business model has produced trillions in revenue and has become one of the fiercest battlegrounds across the internet . Text search allows users to quickly find specific information within a large set of data by entering keywords or phrases. When a query is entered, the text search engine scans through indexed documents to locate and retrieve the most relevant results based on the keywords. Text search is a good solution for queries requiring exact matches where the overarching meaning isn't as critical. Some of the most common uses include: Catalog and content search: Using the search bar to find specific products or content based on keywords from customer inquiries. For example, a customer searching for "size 10 men trainers" or “installation guide” can instantly find the exact items they’re looking for, like how Nextar tapped into Atlas Search to enable physical retailers to create online catalogs Covid-19 pandemic. In-application search: This is well-suited for organizations with straightforward offerings to make it easier for users to locate key resources, but that don’t require advanced features like semantic retrieval or contextual re-ranking. For instance, if a user searches for "songs key of G," they can quickly receive relevant materials. This streamlines asset retrieval, allowing users to focus on the task they are trying to achieve and boosts overall satisfaction. For a company like Yousician , Atlas Search enabled their 20 million monthly active users to tackle their music lessons with ease. Customer 360: Unifying data from different sources to create a single, holistic view. Consolidated information such as user preferences, purchase history, and interaction data can be used to enhance business visibility and simplify the management, retrieval, and aggregation of user data. Consider a support agent searching for all information related to customer “John Doe." They can quickly access relevant attributes and interaction history, ensuring more accurate and efficient service. Helvetia was able to achieve success after migrating to MongoDB and using Atlas Search to deliver a single, 360-degree real-time view across all customer touchpoints and insurance products. AI and a new paradigm with vector search With advances in technology, vector search has emerged to help solve the challenge of providing relevant results even when the user may not know what they’re looking for. Vector search allows you to take any type of media or content, convert it into a vector using machine learning algorithms, and then search to find results similar to the target term. The similarity aspect is determined by converting your data into numerical high-dimensional vectors, and then calculating the distance between them to determine relevance—the closer the vector, the higher the relevance. There is a wide range of practical, powerful use cases powered by vector search—notably semantic search and retrieval-augmented generation (RAG) for gen AI. Semantic search focuses on meaning and prioritizes user intent by deciphering not just what users type but why they're searching, in order to provide more accurate and context-oriented search results. Some examples of semantic search include: Content/knowledge base search: Vast amounts of organizational data, structured and unstructured, with hidden insights, can benefit significantly from semantic search. Questions like “What’s our remote work policy?” can return accurate results even when the source materials do not contain the “remote” keyword, but rather have “return to office” or “hybrid” or other keywords. A real-world example of content search is the National Film and Sound Archive of Australia , which uses Atlas Vector Search to power semantic search across petabytes of text, audio, and visual content in its collections. Recommendation engines: Understanding users’ interests and intent is a strong competitive advantage–like how Netflix provides a personalized selection of shows and movies based on your watch history, or how Amazon recommends products based on your purchase history. This is particularly powerful in e-commerce, media & entertainment, financial services, and product/service-oriented industries where the customer experience tightly influences the bottom line. A success story is Delivery Hero , which leverages vector search-powered real-time recommendations to increase customer satisfaction and revenue. Anomaly detection: Identifying and preventing fraud, security breaches, and other system anomalies is paramount for all organizations. By grouping similar vectors and using vector search to identify outliers, potential threats can be detected early, enabling timely responses. Companies like VISO TRUST and Extrac are among the innovators that build their core offerings using semantic search for security and risk management. With the rise of large language models (LLMs), vector search is increasingly becoming essential in gen AI application development. It augments LLMs by providing domain-specific context outside of what the LLMs “know,” ensuring relevance and accuracy of the gen AI output. In this case, the semantic search outputs are used to enhance RAG. By providing relevant information from a vector database, vector search helps the RAG model generate responses that are more contextually relevant. By grounding the generated text in factual information, vector search helps reduce hallucinations and improve the accuracy of the response. Some common RAG applications are for chatbots and virtual assistants, which provide users with relevant responses and carry out tasks based on the user query, delivering enhanced user experience. Two real-world examples of such chatbot implementations are from our customers Okta and Kovai . Another popular application is using RAG to help generate content like articles, blog posts, scripts, code, and more, based on user prompts or data. This significantly accelerates content production, allowing organizations including Novo Nordisk and Scalestack to save time and produce content at scale, all at an accuracy level that was not possible without RAG. Beyond RAG, an emerging vector search usage is in agentic systems . Such a system is an architecture encompassing one or more AI agents with autonomous decision-making capabilities, able to access and use various system components and resources to achieve defined objectives while adapting to environmental feedback. Vector search enables efficient and semantically meaningful information retrieval in these systems, facilitating relevant context for LLMs, optimized tool selection, semantic understanding, and improved relevance ranking. Hybrid search: The best of both worlds Hybrid search combines the strengths of text search with the advanced capabilities of vector search to deliver more accurate and relevant search results. Hybrid search shines in scenarios where there's a need for both precision (where text search excels) and recall (where vector search excels), and where user queries can vary from simple to complex, including both keyword and natural language queries. Hybrid search delivers a more comprehensive, flexible information retrieval process, helping RAG models access a wider range of relevant information. For example, in a customer support context, hybrid search can ensure that the RAG model retrieves not only documents containing exact keywords but also semantically similar content, resulting in more informative and helpful responses. Hybrid search can also help reduce information overload by prioritizing the most relevant results. This allows RAG models to focus on processing and understanding the most critical information, leading to faster, more accurate responses, and improving the user experience. Powering your AI and search applications with MongoDB As your organization continues to innovate in the rapidly evolving technology ecosystem, building robust AI and search applications supporting customer, employee, and stakeholder experiences can deliver powerful competitive advantages. With MongoDB, you can efficiently deploy full-text search , vector search , and hybrid search capabilities. Start building today—simplify your developer experience while increasing impact in MongoDB’s fully-managed, secure vector database, integrated with a vast AI partner ecosystem , including all major cloud providers, generative AI model providers, and system integrators. Head over to our quick-start guide to get started with Atlas Vector Search today.
Saving Energy, Smarter: MongoDB and Cedalo for Smart Meter Systems
The global energy landscape is undergoing a significant transformation, with energy consumption rising 2.2% in 2023, surpassing the 2010-2019 average of 1.5% per year. This increase is largely due to global developments in BRICS member countries—Brazil, Russia, India, China, and South Africa. As renewable sources like solar power and wind energy become more prevalent (in the EU, renewables accounted for over 50% of the power mix in the first quarter of 2024 ), ensuring a reliable and efficient energy infrastructure is crucial. Smart meters, the cornerstone of intelligent energy networks, play a vital role in this evolution. According to IoT analyst firm Berg Insight, the penetration of smart meters is skyrocketing, with the US and Canada expected to reach nearly 90% adoption by 2027, whereas China is expected to account for as much as 70–80% of smart electricity meter demand across Asia in the next few years. This surge is indicative of a growing trend towards smarter, more sustainable energy solutions. In Central Asian countries, the Asian Development Bank is supporting the fast deployment of smart meters to save energy and improve the financial position of countries' power utilities. This article will delve into the benefits of smart meters, the challenges associated with managing their data, and the innovative solutions offered by MongoDB and Cedalo. The rise of smart meters Smart meters, unlike traditional meters that require manual readings, collect and transmit real-time energy consumption data directly to energy providers. This digital transformation offers numerous benefits, including: Accurate Billing: Smart meters eliminate the need for estimations, ensuring that consumers are billed precisely for the energy they use. Personalized Tariffs: Energy providers can offer tailored tariffs based on individual consumption patterns, allowing consumers to take advantage of off-peak rates, special discounts, and other cost-saving opportunities. Enhanced Grid Management: Smart meter data enables utilities to optimize grid operations, reduce peak demand, and improve overall system efficiency. Energy Efficiency Insights: Consumers can gain valuable insights into their energy usage patterns, identifying areas for improvement and reducing their overall consumption. With the increasing adoption of smart meters worldwide, there is a growing need for effective data management solutions to harness the full potential of this technology. Data challenges in smart meter adoption Despite the numerous benefits, the widespread adoption of smart meters also presents significant data management challenges. To use smart metering, power utility companies need to deploy a core smart metering ecosystem that includes the smart meters themselves, the meter data collection network, the head-end system (HES), and the meter data management system (MDMS). Smart meters collect data from end consumers and transmit it to the data aggregator via the Local Area Network (LAN). The transmission frequency can be adjusted to 15 minutes, 30 minutes, or hourly, depending on data demand requirements. The aggregator retrieves the data and then transmits it to the head-end system. The head-end system analyzes the data and sends it to the MDMS. The initial communications path is two-way, signals or commands can be sent directly to the meters, customer premise, or distribution device. Figure 1: End-to-end data flow for a smart meter management system / advanced metering infrastructure (AMI 2.0) When setting up smart meter infrastructure, power, and utility companies face several significant data-related challenges: Data interoperability: The integration and interoperability of diverse data systems pose a substantial challenge. Smart meters must be seamlessly integrated with existing utility systems and other smart grid components often requiring extensive upgrades and standardization efforts. Data management: The large volume of data generated by smart meters requires advanced data management and analytics capabilities. Utilities must implement robust data storage, processing, and analysis solutions to handle real-time time series data streams storage, analysis for anomaly detection, and trigger decision-making processes. Data privacy: Smart meters collect vast amounts of sensitive information about consumer energy usage patterns, which must be protected against breaches and unauthorized access. Addressing these challenges is crucial for the successful deployment and operation of smart meter infrastructure. MQTT: A cornerstone of smart meter communication MQTT , a lightweight publish-subscribe protocol, shines in smart meter communication beyond the initial connection. It's ideal for resource-constrained devices on low-bandwidth networks, making it perfect for smart meters. While LoRaWAN or PLC handle meter-to-collector links, MQTT bridges Head-End Systems (HES) and Meter Data Management Systems (MDMS). Its efficiency, reliable delivery, and security make it well-suited for large-scale smart meter deployments. Cedalo MQTT platform and MongoDB: A powerful combination Cedalo , established in 2017, is a leading German software provider specializing in MQTT solutions. Their flagship product, the Cedalo MQTT Platform, offers a comprehensive suite of features, including the Pro Mosquitto MQTT broker and Management Center . Designed to meet the demands of large enterprises, the platform delivers high availability, audit trail logging, persistent queueing, role-based access control, SSO integration, advanced security, and enhanced monitoring. To complement the platform's capabilities, MongoDB's Time Series collections provide a robust and optimized solution for storing and analyzing smart meter data. These collections leverage a columnar storage format and compound secondary indexes to ensure efficient data ingestion, reduced disk usage, and rapid query processing. Additionally, window functions enable flexible time-based analysis, making them ideal for IoT and analytical applications. Figure 2: MongoDB as the main database for the meter data management system where it receives meter data via Pro Mosquitto MQTT broker. Let us revisit Figure 1 and leverage both the Cedalo MQTT Platform and MongoDB in our design. In Figure 2, the Head-end System (HES) can use MQTT to filter, aggregate, and convert data before storing it in MongoDB. This data flow can be established using the MongoDB Bridge plugin provided by Cedalo. Since the MQTT payload is JSON, it is ideal to store it in MongoDB as the database stores data in BSON (Binary JSON). The MongoDB Bridge plugin offers advanced features such as flexible data import settings (specifying target databases and collections, choosing authentication methods, and selecting specific topics and message fields to import) and advanced collection mapping (mapping multiple MQTT topics to one or more collections with the ability to choose specific fields for insertion). MongoDB's schema flexibility is crucial for adapting to the ever-changing structures of MQTT payloads. Unlike traditional databases, MongoDB accommodates shifts in data format seamlessly, eliminating the constraints of rigid schema requirements. This helps with interoperability challenges faced by utility companies. Once the data is stored in MongoDB, it can be analyzed for anomalies. Anomalies in smart meter data can be identified based on various criteria, including sudden spikes or drops in voltage, current, power, or other metrics that deviate significantly from normal patterns. Here are some common types of anomalies that we might look for in smart meter data: Sudden spikes or drops: These include voltage, current, or power spikes or drops. A sudden increase or decrease in voltage beyond expected limits. Outliers: Data points that are significantly different from the majority of the data. Unusual patterns: Unusually high or low energy consumption compared to historical data or inconsistent power factor readings. Frequency anomalies: Frequency readings that deviate from the normal range. MongoDB's robust aggregation framework can aid in anomaly detection. Both anomaly data and raw data can be stored in time series collections, which offer reduced storage footprint and improved query performance due to an automatically created clustered index on timestamp and _id. The high compression offered addresses the challenge of data management at scale. Additionally, data tiering capabilities like Atlas Online Archive can be leveraged to push cold data into cost-effective storage. MongoDB also provides built-in security controls for all your data, whether managed in a customer environment or MongoDB Atlas, a fully managed cloud service. These security features include authentication, authorization, auditing, data encryption (including Queryable Encryption ), and the ability to access your data security with dedicated clusters deployed in a unique Virtual Private Cloud (VPC). End-to-end solution Figure 3: End-to-end data flow Interested readers can clone this repository and set up their own MongoDB-based smart meter data collection and anomaly detection solution. The solution follows the pattern illustrated in Figure 3, where a smart meter simulator generates raw data and transmits it via an MQTT topic. A Mosquitto broker receives these messages and then stores them in a MongoDB collection using the MongoDB Bridge. By leveraging MongoDB change streams , an algorithm can retrieve these messages, transform them according to MDMS requirements, and perform anomaly detection. The results are stored in a time series collection using a highly compressed format. The Cedalo MQTT Platform with MongoDB offers all the essential components for a flexible and scalable smart meter data management system, enabling a wide range of applications such as anomaly detection, outage management, and billing services. This solution empowers power distribution companies to analyze trends, implement real-time monitoring, and make informed decisions regarding their smart meter infrastructure. We are actively working with our clients to solve IoT challenges. Take a look at our Manufacturing and Industrial IoT page for more stories.
The Dual Journey: Healthcare Interoperability and Modernization
Interoperability in healthcare isn’t just a buzzword; it’s a fundamental necessity. It refers to the ability of IT systems to enable the timely and secure access, integration, and use of electronic health data. However, integrating data across different applications is a pressing challenge, with 48% of US hospitals reporting a one-sided sharing relationship in which they share patient data with other providers who do not, in turn, share patient data with the hospital. The ability to share electronic health data seamlessly across various healthcare systems can revolutionize patient care, enhance operational efficiency, and drive innovation. In this post, we’ll explore the challenges of healthcare data sharing, the role of interoperability , and how MongoDB can be a game-changer in this landscape. The challenge of data sharing in healthcare Today's consumers have high expectations for accessing information, and many now anticipate quick and continuous access to their health and care records. One of the biggest IT challenges faced by healthcare organizations is sharing data effectively and creating seamless data integrations to build patient-centric healthcare solutions. Healthcare data has to be shared in multiple ways: Between internal applications to ensure seamless data flow across various internal systems. Between primary and secondary care , to coordinate care across healthcare providers. To patient portals and telemedicine to enhance patient engagement and remote care. To payers, institutions, and patients themselves , to streamline interactions with insurance companies and regulatory bodies. To R&D units to accelerate medical research and pharmaceutical developments. The complexity of healthcare data is staggering, and hospitals regularly need to integrate dozens of different applications—all of which means that there are significant barriers to healthcare data sharing and integration. A vision for patient-centric healthcare Imagine a world where patient data is shared in real-time with all relevant parties—doctors, hospitals, labs, pharmacies, and insurance companies. This level of interoperability would streamline the flow of information, reduce errors, and improve patient outcomes. Achieving this, however, is no easy feat as healthcare data is immensely complex, involving various types of data such as unstructured clinical notes, lab tests, medical images, medical devices, and even genomic data. Furthermore, types of data mean different things depending on where and where it was collected. Achieving seamless data sharing also involves overcoming barriers to data sharing between different healthcare providers and systems, all while adapting to evolving regulations and standards. Watch the " MongoDB and FHIR: Navigating Healthcare Data " session from MongoDB.local NYC on YouTube. The intersection of modernization and interoperability Modernization of healthcare IT systems and achieving interoperability are two sides of the same coin. Both require significant investments and a focus on transitioning from application-driven to data-driven architecture. By focusing first on data and then connecting applications with a developer data platform like MongoDB Atlas , healthcare organizations can avoid data silos and achieve vendor-neutral data ownership. As healthcare interoperability standards define a common language, organizations might question whether, instead of reinventing the wheel with their own data domains, they can use the interoperability journey (and its high investments) to modernize their applications. MongoDB’s document data model supports the JSON format, just like FHIR (Fast Healthcare Interoperability Resources) and other interoperability standards, making it a more efficient and flexible data platform for developing healthcare applications beyond the limitations of external APIs. FHIR for storing healthcare data? The most implemented standard worldwide, HL7 FHIR , treats each piece of healthcare data as a self-contained resource, which can be accessed through unique URLs on FHIR servers, similar to how web pages are accessed on the internet. FHIR adopted a pragmatic approach for which there was no need to define a complete and unique set of resources for all the clinical data, but trying to get what most of the electronic health records (EHR) share. That is the 80% / 20% approach. For the 20% of non-standardized data, they created the famous FHIR Extensions , to extend every resource to the specific needs. However, FHIR is not yet fully developed, with only 15 of the 158 resources it defines having reached the highest level of maturity. The constant changes can be as simple as a name change or can be so complex that data has to be rearranged. FHIR is designed for exchange of data, however, as questioned above, it might be thought to be used also for persistence. In this post from Alastair Allen, he discusses the problems that could arise when using FHIR for persistence depending on the use case, and this chart aids in the decision-making. Figure 1: Using FHIR for persistence depending on the complexity of the use case For specific applications with no complex data, such as an Enterprise Master Patient Index (EMPI) to match and unify patient demographics across applications or to integrate data from wearables, leveraging FHIR for persistence is a good approach. However, building a primary operational repository for broader applications like a Patient Summary or even a Healthcare Information System (HIS) presents a significant challenge. This is because the data model these applications require for a specific organization goes beyond the capabilities of FHIR, and solving the bias through FHIR extensions makes this an inefficient approach. OpenEHR as an alternative approach In Catalonia, Spain—for about 8 million people and roughly 60 public hospitals—there are 29 different hospital EHR systems. This fact converts Catalonia to a long-experienced actor in healthcare interoperability. Each hospital maintains a team of developers exclusively focused on building interoperability interfaces. This is not a one-time cost, but it is a fact that there will be an increasing demand for data sharing. This will never end, and the costs are increasingly higher. In Catalonia, as well as in other European regions, some thought about changing the paradigm. Rather than implementing interoperability interfaces, why not create new applications that are implicitly interoperable? This is what openEHR proposes, defining the clinical information model from the maximal perspective and developing applications that consume a subset of the clinical information system. This is what the openEHR fans relate to as “Interoperability.” It’s not a protocol but an open architecture. However, while FHIR is very versatile—offering data models for administrative and operational data efficiently—openEHR focuses exclusively on clinical data. So while a combination of FHIR and openEHR can solve part of the problem, future healthcare applications need to integrate a wide variety of data, including medical images, genomics, proteomics, and data from complex medical devices—which could be complicated by the lack of a single standard. Overcoming this challenge with the document model and MongoDB Now, let’s discover the power of the document model to advance interoperability while modernizing systems. MongoDB Atlas features a flexible document data model, which provides a flexible way of storing and organizing healthcare data using JSON-like documents. With a flexible data schema, healthcare organizations can accommodate any data structure, format, or source into one platform, providing seamless third-party integration capabilities necessary for interoperability. While different use cases will have different solutions, the flexibility of the document model means MongoDB is able to adapt to changes. Figure 2 below shows a database modeled in MongoDB, where each collection stores a FHIR resource type (e.g., patients, encounters, conditions). These documents mirror the FHIR objects, conserving its complex hierarchy. Let's imagine our application requires specific fields not supported by FHIR, and there is no need for an FHIR extension because it won’t be shared externally. We can add a metadata field containing all this information that is as flexible as needed. It can be used to track the standard evolution of the resource, the version of the document itself, tenant ID for multi-tenant applications, and more. Figure 2: Data modeled in MongoDB Another possibility is to add the searchable fields of the resource as key-value pairs so that you can retrieve data with a single index. We can maintain the indexes by automating this with the FHIR search parameters. In a single repository, we can combine FHIR data with custom application data. Additionally, data in other protocols, such as DICOM metadata and OMOP, can be integrated, providing unparalleled flexibility in accessing your data. This setup permits access through different endpoints within one repository. Using MQL (the MongoDB Query Language), you can build FHIR APIs or use the MongoDB SQL interface to provide SQL querying capabilities to connect your preferred business intelligence tools. Figure 3: Unified and flexible data store and flexible data retrieval MongoDB’s developer data platform At the center of MongoDB’s developer data platform is MongoDB Atlas, the most advanced cloud database service on the market. It provides integrated full-text search capabilities, allowing applications to perform when making complex queries without the need to maintain a separate system. With all the possibilities of generative AI, multi-dimensional vectors to represent the meaning of the data are becoming a necessity for all vendors and organizations. MongoDB’s capabilities of storing vectors alongside the operational data while providing the vector search aggregation stage functionality, enables fast data retrieval with the possibility of pre-filtering your data with any of the data existing in your operational database. This is executed in a single query, deriving in a best-of-class performance operation. Figure 4: Vector embeddings These capabilities transform a single database into a unique, powerful, and easy-to-use interface capable of handling diverse use cases without the need for any single-purpose databases. Solving the dual challenge with MongoDB Achieving interoperability and modernization in healthcare IT are challenging but essential. MongoDB provides a powerful platform that meets organizations’ modern data management needs. By embracing MongoDB, healthcare organizations can unlock the full potential of their data, leading to improved patient outcomes and operational efficiency. Figure 5: Closing the gap between interoperability and modernization journey with MongoDB Refactoring applications to incorporate interoperability resources as part of documents—and extending them with all the requirements for your modern needs—will ensure organizations’ data layers remain robust and adaptable. By doing so, organizations can create a flexible architecture that can seamlessly integrate diverse data types and accommodate future advancements. This approach not only enhances data accessibility and simplifies data management but also supports compliance with evolving standards and regulations. Furthermore, it enables real-time data analytics and insights, fostering innovation and driving better decision-making. Ultimately, this strategy positions healthcare organizations to effectively manage and leverage their data, leading to improved patient outcomes and operational efficiencies. For more detailed information and resources on how MongoDB can transform organizations’ healthcare IT systems, we encourage you to apply for an exclusive innovation workshop with MongoDB's industry experts to explore bespoke modern app development and tailored solutions for your organization. Additionally, check out these resources: MongoDB and FHIR: Navigating Healthcare Data with MongoDB How Leading Industries are Transforming with AI and MongoDB Atlas The MongoDB Solutions Library is curated with tailored solutions to help developers kick-start their projects For developers: From FHIR Synthea data to MongoDB
CTF Life Leverages MongoDB Atlas to Deliver Customer-Centric Service
Hong Kong-based Chow Tai Fook Life Insurance Company Limited (CTF Life) is proud of its rich, nearly 40-year history of providing a wide range of insurance and financial planning services. The company provides life, health, accident, savings, and investment insurance to its customers, helping them and their loved ones navigate life’s journey with personalized planning solutions, lifelong protection, and diverse lifestyle experiences. A wholly-owned subsidiary of NWS Holdings Limited and a member of Chow Tai Fook Group, CTF Life consistently strengthens its collaboration with the diverse conglomerate of the Cheng family (Chow Tai Fook Group) and draws on the Group’s robust financial strength, strategic investments across the globe, and advanced customer-focused digital technology with the aspiration of becoming a leading insurance company in the Greater Bay Area. To achieve this goal, CTF Life modernized its on-premises infrastructure to provide the speed and flexibility required to offer customers personalized experiences. To turn their vision into reality, CTF Life decided to adopt MongoDB Atlas . By modernizing their systems and processes with the world’s most versatile developer data platform, CTF Life knew they’d be able to meet customer expectations, offering improved customer service, faster response times, and more convenient access to their products and services. Data-driven customer service The insurance industry is undergoing a significant shift, from traditional data management to near-real-time data-driven insights, driven by strong consumer demand and the urgent need for companies to process large amounts of data efficiently. As insurance companies strive to provide personalized and real-time products, the move towards sophisticated and real-time data-driven customer service is inevitable. CTF Life is on its digital transformation journey to modernize its relational database management system (RDBMS) infrastructure to empower its agents, known as Life Planners, to provide enhanced customer experiences. The company faced obstacles to legacy systems and siloed data. Life Planners were spending a lot of time looking up customer information from various systems and organizing this into useful customer insights. Not having a holistic view of customer data also made it challenging to recommend personalised products and services within CTF Life, the Group, and beyond. Reliance on legacy RDBMS systems presented a major challenge in CTF Life’s pursuit of leveraging real-time customer information to enhance customer experiences and operational efficiency. For their modernization efforts, CTF Life was looking for the following required capabilities: A modernized application with agile development No downtime for changing schema, new modules, or feature updates A single way of centralizing and organizing data from a number of sources (backend, CRM, etc.) into a standardized format ready for a front-end mobile application A future-proof data platform with extensible capability for analytics across CTF Life, their diverse conglomerate collaboration, and their strategic partners to support the company’s digital solutions Embracing the operational data layer for enhanced experiences CTF Life knew they had to build a solution for the Life Planners to harness the wealth of useful information available to them, making it easier to engage and connect with customers. The first project identified was their clienteling system, which is designed to establish long-term relationships with customers based on data about their preferences, behaviors, and needs. To overcome their legacy systems and siloed data, CTF Life built their clienteling system on MongoDB Atlas . Atlas serves as the digital data store for Life Planners, creating a single view of the customer (SVOC) with a flexible document model that enables CTF Life to handle large volumes of customer data in real-time efficiently. By integrating their operational data into one platform with MongoDB Atlas on Microsoft Azure, CTF Life’s revamped clienteling system provides their Life Planners with a comprehensive view of customer profiles, which allows them to share targeted content with customers. Additionally, CTF Life is using Atlas Search to build relevance-based search capabilities directly into the application, making it faster and easier to search for customer data across the company’s system landscape. These benefits helped improve customer service with faster access to data with an SVOC so Life Planners can provide more accurate and timely information to their customers. Atlas Search is now the foundation of the clienteling system, which powers data analytics and machine learning capabilities to support various use cases. For example, the clienteling app's smart reminder feature recognizes key moments in a customer's life, like the impending arrival of a newborn child. Based on these types of insights, the app can help Life Planners make personalized recommendations to the customer about relevant services and products that may be of interest to them as new parents. Because of its work with MongoDB, CTF Life can now analyze customer profiles and use smart reminders to engage customers at the right time in the right context. This has made following up with customers and leads faster and easier. And, contacting prospects, scheduling appointments, setting reminders, sharing relevant content, running campaigns and promotions, recommending products and services, and tracking lead progress can all be performed in one system. Moreover, access to real-time data enables Life Planners to streamline their work and reduce manual processes. And data-driven insights empower Life Planners to make informed decisions quickly. They can analyze customer information, identify trends, and tailor their recommendations to meet individual needs more effectively. With MongoDB Atlas Search, Life Planners can use advanced search capabilities to identify opportunities to serve customers better. Continuing to create value beyond insurance CTF Life strives to provide its customers with value beyond insurance. Through a range of collaborations with Chow Tai Fook Group, and strategic partnerships with technology partners like MongoDB, CTF Life has created a customer-centric approach and continues to advance its digital transformation strategy to enhance a well-rounded experience for customers that goes beyond insurance with a sincere and deep understanding of their diverse needs in every chapter of their life journey. In the future, CTF Life will continue to build upon its strategic partnership with MongoDB and expand the use of its digital data store on MongoDB Atlas by creating new client servicing modules on the mobile app their Life Planners use. CTF Life will also be expanding its search capabilities with Atlas Vector Search to accelerate their journey to building advanced search and generative AI applications for more automated servicing. Partnering with MongoDB helped us prioritize technology that accelerates our digital transformation. The integration between generative AI and MongoDB as a medium for information search can be leveraged to further support front-line Life Planners as well as mid/back-office operations. Derek Ip, Chief Digital and Technology Officer of CTF Life Learn how to tap into real-time data with MongoDB Atlas .
Built With MongoDB: Atlas Helps Team-GPT Launch in Two Weeks
Team-GPT enables teams large and small to collaborate on AI projects. When OpenAI released GPT-4, it turned out to be a game-changer for the startup. Founded in 2023, the company has been helping people train machine learning (ML) models, in particular natural language processing (NLP) models. But when OpenAI launched GPT-4 in March 2023, the team was blown away by how much progress had been made on large language models (LLMs). So Team-GPT dropped everything they were doing and started experimenting with it. Many of those early ideas are still memorialized on a whiteboard in one of the office's meeting rooms: The birth of an idea. Like many startups, Team-GPT began with a brainstorm on a whiteboard. Evolving the application Of all the ideas they batted around, there was one issue in particular the team wanted to solve—the need for a shared workspace where they could experiment with LLMs together. What they found was that having to work with LLMs in the terminal was a major point of friction. Plus, there weren't any sharing abilities. So they set out to create a UI consisting of chat sharing, in-chat team collaboration, folders and subfolders, and a prompt library. The whole thing came together in an incredibly short period of time. This was due, in large part, to their initial choice of MongoDB Atlas, which allowed them to build with speed and scalability. "MongoDB made it possible for us to launch in just two weeks," said Team-GPT Founder and CTO, Ilko Kacharov. "With the MongoDB Atlas cloud platform, we were able to move rapidly, focusing our efforts on developing innovative product features rather than dealing with the complexities of infrastructure management." Before long, the team realized there was a lot more that could be built around LLMs than simply chat, and set out to add more advanced capabilities. Today, users can integrate any LLM of their choice and add custom instructions. The platform also supports multimodality like ChatGPT Vision and DALL-E. Users use any GPT model to turn chat responses into a standalone document that can then be edited. All these improvements are meant to unify teams' AI workflows in a single, AI-powered tool. A platform built for developers Diving deeper into more technical aspects of the solution, Team-GPT CEO Iliya Valchanov acknowledges the virtues of the document data model, which underpins the Atlas developer data platform. "We wanted the ability to quickly update and create new collections, add more data, and expand the existing database setup without major hurdles or time consumption," he said. "That's something that relational databases often struggle with." A developer data platform consists of integrated data infrastructure components and services for quick deployment. With transactional, analytical, search, and stream processing capabilities, it supports various use cases, reduces complexity, and accelerates development. Valchanov's team leverages a few key elements of the platform to address a range of application needs. "We benefited from Atlas Triggers , which allow automatic execution of specified database operations," he said. "This greatly simplified many of our routine tasks." It's not easy to build truly differentiated applications without a friction-free developer experience. Valchanov cites Atlas' user-friendly UI as a key advantage for a startup where time is of the essence. And he said that Atlas Charts has been instrumental for the team, who use it every day, even their less technical people. Of course one of the biggest reasons why developers and tech leaders choose MongoDB, and why so many are moving away from relational databases, is its ability to scale—which Valchanov said is one of the most critical requirements for supporting the company's growth. "With MongoDB handling the scaling aspect, we were able to focus our attention entirely on building the best possible features for our customers." Team-GPT deployment options Accelerating AI transformation Team-GPT is a collaborative platform that allows teams of up to 20,000 people to use AI in their work. It's designed to help teams learn, collaborate, and master AI in a shared workspace. The platform is used by over 2,000 high-performing businesses worldwide, including EY, Charles Schwab, Johns Hopkins University, Yale University, and Columbia University, all of which are also MongoDB customers. The company's goal is to empower every person who works on a computer to use AI in a productive and safe manner. Valchanov fully appreciates the rapid change that accompanies a product's explosive growth. "We never imagined that we would eventually grow to provide our service to over 40,000 users," he said. "As a startup, our primary focus when selecting a data platform was flexibility and the speed of iteration. As we transitioned from a small-scale tool to a product used by tens of thousands, MongoDB's attributes like flexibility, agility, and scalability became necessary for us." Another key enabler of Team-GPT's explosive growth has been the MongoDB for Startups program . It offers valuable resources such as free Atlas credits, technical guidance, co-marketing opportunities, and access to a network of partners. Valchanov makes no secret of how instrumental the program has been for his company's success. "The startup program made it free! It offered us enough credits to build out the MVP and cater to all our needs," he said. "Beyond financial aid, the program opened doors for us to learn and network. For instance, my co-founder, Yavor Belakov, and I participated in a MongoDB hackathon in MongoDB's office in San Francisco." Team-GPT co-founders Yavor Belakov (l) and Iliya Valchanov (r) participated in a MongoDB hackathon at the San Francisco office Professional services engagements are an essential part of the program, especially for early-stage startups. "The program offered technical sessions and consultations with MongoDB staff, which enriched our knowledge and understanding, especially for Atlas Vector Search , aiding our growth as a startup," said Valchanov. The roadmap ahead for the company includes the release of Team-GPT 2.0, which will introduce a brand-new user interface and new, robust functionalities. The company encourages anyone looking to learn more or join their efforts to ease adoption of AI innovations to reach out on LinkedIn . Are you part of a startup and interested in joining the MongoDB for Startups program? Apply to the program now . For more startup content, check out our Built With MongoDB blog collection.
Agnostiq & MongoDB: High-Performance Computing for All
Material scientists, computational biologists, and AI researchers all have at least one thing in common; the need for huge amounts of processing power, or ‘compute’, to turn raw data into results. But here’s the problem. Many researchers lack the skills needed to build the workflows that move data through huge networks of distributed servers, CPUs, and GPUs that actually do the number crunching. And that’s where Agnostiq comes in. Since the company’s inception in 2018, Agnostiq has put the power of high-performance computing (HPC) in the hands of researchers, bypassing the need for development expertise to build these essential data and compute pipelines. Power to the people “We started research on high-performance computing needs in fields like finance and chemistry, and through the process of onboarding, researchers quickly realized how hard it was for [researchers] to access and scale up on the cloud, or tap into HPC and GPU resources,'' said Santosh Kumar Radha, Agnostiq’s Head of Product. “If you wanted to scale up, there were not many tools available in the major cloud providers to do this.” To address this bottleneck, the team at Agnostiq built Covalent, a Python-based framework that allows researchers to easily design and run massive compute jobs on cloud platforms, on-prem clusters, and HPC services. With Covalent, startups and enterprises can build any AI or HPC application in a simple, scalable, and cost-effective way using a Python notebook, negating the need to interact with underlying infrastructure. One of the hardest challenges the Covalent team faced was combining traditional HPC with modern cloud technology. Because traditional HPC infrastructure was never designed to run in the cloud, the team spent considerable resources marrying techniques like GPU and CPU parallelization, task parallelization, and graph optimization with distributed cloud computing environments. As a result, researchers can use Covalent to quickly create a workflow that combines the convenience of cloud computing with specialized GPU providers and other HPC services. Everything, everywhere, all at once As the name suggests, Agnostiq has always focused on making their platform as open and resource neutral as possible. MongoDB Atlas , with its native multi-cloud capability, was a perfect complement. “At Agnostiq, everything we build has to be technology and vendor neutral. Interoperability is key for us,” said Radha. “We do all the mapping for our customers, so our platform has to perform a seamless transition from cloud to cloud.” The ability to move data between clouds became even more critical following the release of ChatGPT. With an explosion in generative AI research and development, the availability of GPU resources plummeted. “Resource scarcity in the ‘GPT era’ means you couldn’t get access to GPUs anywhere,” Radha added. “If you didn’t have a default cloud posture, you were nowhere, which is why we doubled down on multi-cloud and MongoDB Atlas to give our clients that optionality.” Open source opening doors Since the beginning, the team at Agnostiq has chosen MongoDB as their default NoSQL database. At first, the team adopted MongoDB’s free, open source product. “We didn’t have any DBAs as a small agile team. MongoDB gave us the freedom to build and manage our data workflows without the need for a specialist,” said William Cunningham, Head of HPC at Agnostiq. As their customer base grew along with the demand for cloud computing access, Agnostiq moved to MongoDB Atlas, gaining the freedom to move data seamlessly between AWS, Google Cloud, and Microsoft Azure. This gave Covalent the flexibility to reach multi-cloud compatibility at a faster rate than with standard tooling. Covalent provides a workflow management service by registering jobs, dispatching IDs, and collecting other metadata that allows fellow researchers and developers to reproduce the original work. MongoDB is used in the front-end, allowing a high volume of metadata and other assets to be published and cached in accordance with an event-driven architecture. This near real-time experience is key to a product aimed at delivering a unified view over distributed resources. MongoDB Atlas further provided the autoscaling required to grow with the user base and the number of workloads while keeping costs in check. “MongoDB Atlas helps us provide an ideal foundation for modern HPC and AI applications which require serverless compute, autoscaling resources, distributed workloads, and rapidly reconfigurable infrastructure,” added Radha. The future Looking to the future, Agnostiq is focused on servicing the huge demand for gen AI modeling and workflow building. To that end, the company released its own inference service called Function Serve within Covalent. Function Serve offers customers a complete, enterprise-grade solution for AI development and deployment, supporting serverless AI model training and fine-tuning. With Function Serve, customers can fine-tune, host, and serve any open-source or proprietary model with full infrastructure abstraction, all with only a few additional lines of code. MongoDB Atlas was used to rapidly develop a minimal service catalog while remaining cloud-agnostic. Looking ahead, the team plans to leverage MongoDB Atlas for enterprise and hybrid-cloud deployments in order to quickly meet customers in their existing cloud platforms. Agnostiq is a member of the MongoDB AI Innovators program , providing their team with access to Atlas credits and technical best practices. You can get started with your AI-powered apps by registering for MongoDB Atlas and exploring the tutorials available in our AI resources center . Additionally, if your company is interested in being featured, we'd love to hear from you. Reach out to us at ai_adopters@mongodb.com .
Meeting the UK’s Telecommunications Security Act with MongoDB
Emerging technologies like AI, IoT, and 5G have transformed the value that telecommunications companies provide the world. However, these new technologies also present new security challenges. As telcos continue to amass large amounts of sensitive data, they become an increasingly attractive target for cybercriminals — making both companies and countries vulnerable to cyberattacks. Fortunately, developers can protect user data which comes with strong security requirements on a developer data platform. By offering features to meet stringent requirements with robust operational and security controls, telcos can protect their customers’ private information. The UK Telecommunications Security Act Amid growing concerns about the vulnerability of telecom infrastructure, and its increasing digital dependency, the UK Telecommunications (Security) Act (TSA) was enacted on November 17, 2021. It was designed to bolster the security and resilience of the UK’s telecommunications networks. The TSA mandates that telecom operators implement rigorous security measures such as end-to-end encryption as well as identity and access management to protect their networks from a broad spectrum of threats, ensuring the integrity and continuity of critical communication services. The act allows the government to compel telecom providers to meet specific security directives. The United Kingdom’s Office of Communications (Ofcom) is a regulatory body responsible for overseeing compliance, conducting inspections, and enforcing penalties on operators that fail to meet the standards. The comprehensive code of practice included in the act offers detailed guidance on the security measures that should be implemented, covering risk management, network architecture, incident response, and supply chain security. The TSA tiering system The TSA establishes a framework for ensuring the security of public electronic communications networks and services. It categorizes telecoms providers into different tiers, with specific security obligations for each tier. The Act outlines three main tiers: Tier 1: These are the largest and most critical providers. They have the most extensive obligations due to their significant role in the UK's telecoms infrastructure. Tier 1 providers must comply with the full set of security measures outlined in the Act. Tier 2: These providers have a considerable role in the telecoms network but are not as critical as Tier 1 providers. They have a reduced set of obligations compared to Tier 1 but still need to meet substantial security requirements. Tier 3: These are smaller providers with a limited impact on the overall telecoms infrastructure. Their obligations are lighter compared to Tiers 1 and 2, reflecting their smaller size and impact. The specific obligations for each tier include measures related to network security, incident reporting, and supply chain security. The aim is to ensure a proportional approach to securing the telecoms infrastructure, with the highest standards applied to the most critical providers. Non-compliance may result in fines Under the TSA, non-compliance with security obligations can result in substantial fines. The fines are designed to be significant enough to ensure compliance and deter breaches. The significance of the fines imposed under the TSA underscores the importance the UK government places on telecom security and the serious consequences of failing to meet the established standards. How MongoDB can help MongoDB offers built-in security controls for all your data—whether your databases are managed on-premises with MongoDB Enterprise Advanced or with MongoDB Atlas , our fully managed cloud service. MongoDB enables enterprise-grade security features and simplifies deploying and managing your databases. Encrypting sensitive data The TSA emphasizes securing telecom networks against cyber threats. While specific encryption requirements are not detailed, the focus is on robust security practices, including encryption to protect data integrity and confidentiality. Operators must implement measures that prevent unauthorized access and ensure data security throughout transmission and storage. Compliance may involve regular risk assessments and adopting state-of-the-art technologies to safeguard the network infrastructure. MongoDB data encryption offers robust features to protect your data while it’s in the network, being stored, in memory, in transit (network), at rest (storage), and in use (memory, logs). Customers can use automatic encryption of key data fields like personally identifiable information (PII) or any data deemed sensitive—ensuring data is encrypted through its use. Additionally, with our industry-first Queryable Encryption , MongoDB offers a fast, searchable encryption scheme that supports equality searches, with additional query types such as range, prefix, suffix, and substring planned for future releases. Authentication and Authorization The TSA contemplates stringent identity and access management requirements to enhance network security. Regular audits and reviews of access permissions should be designed to prevent unauthorized access and to quickly identify and respond to potential security breaches. These measures aim to protect the integrity and confidentiality of telecommunications infrastructure. MongoDB enables users to authenticate to their Atlas UI with their Atlas credentials or via single sign-on with their GitHub or Google accounts. Atlas also supports MFA with various options, including OTP authenticators, push notifications, FIDO2 (hardware security keys or biometrics), SMS, and e-mail. MongoDB Enterprise Advanced users can authenticate to the MongoDB database using mechanisms including SCRAM, x.509 certificates, LDAP, OIDC, and passwordless authentication with AWS-IAM. Auditing Under the TSA, providers must implement logging mechanisms to detect and respond to security incidents effectively. Logs should cover access to sensitive systems and data, including unsuccessful access attempts, and must be comprehensive, capturing sufficient detail to facilitate forensic investigations. Additionally, logs should be kept for a specified minimum period and to be protected against unauthorized access, tampering, and loss. MongoDB offers granular auditing that monitors actions in your MongoDB environment and is designed to prevent and detect any unauthorized access to data, including CRUD operations, encryption key management, authentication, role-based access controls, replication, and sharding cluster operations. Additionally, MongoDB’s Atlas Organization Activity Feed displays select events that occurred for a given Atlas organization, such as billing or access events. Likewise, the Atlas Project Activity Feed displays select events that occurred for a given Atlas project. Network security The TSA outlines several network security requirements to ensure the protection and resilience of telecommunications networks. These requirements encompass various aspects of network security, including risk management, protection measures, incident response, and compliance with standards and best practices. Atlas offers many options to securely access your data with dedicated clusters deployed in a unique virtual private cloud (VPC) to isolate your data and prevent inbound network access from the internet. You can also allow a one-way connection from your AWS, Azure, or Google Cloud VPC/VNet to Atlas Clusters via Private Endpoints . Additionally, you can enable peering between your MongoDB Atlas VPC or VNet to your own dedicated application tier VPN with the cloud provider of your choice or enable only specific network segments to connect to your Atlas clusters via the IP Access list . In summary, the UK TSA is a critical regulatory framework aimed at protecting the nation’s telecommunications infrastructure from cyber threats. For telecom companies, compliance isn’t just a legal obligation but a business imperative. Failure to comply can mean significant financial penalties, reputational harm, and long-term operational challenges, underscoring the importance of adopting robust security measures and maintaining continuous adherence to the Act’s requirements. Visit MongoDB’s Strong Security Defaults page for more information on protecting your data with strong security defaults on the MongoDB developer data platform, as well as how to meet stringent requirements with robust operational and security controls.
Leveraging Database Observability at MongoDB: Real-Life Use Case
This post is the second in our three-part series, Leveraging Database Observability at MongoDB. Welcome back to the Leveraging Database Observability at MongoDB series. In our last discussion, we explored MongoDB's unique observability strategy using out-of-the-box tools designed to automatically monitor and optimize customer databases. These tools provide continuous feedback to answer critical questions such as what is happening, where is the issue, why is it occurring, and how do I fix it? This ensures enhanced performance, increased productivity, and minimized downtime. So let’s dive into a real-life use case, illustrating how different tools in MongoDB Atlas come together to address database performance issues. Whether you're a DBA, developer, or just a MongoDB enthusiast, our goal is to empower you to harness the full potential of your data using the MongoDB observability suite. Why is it essential to diagnose a performance issue? Identifying database bottlenecks and pinpointing the exact problem can be daunting and time-consuming for developers and DBAs. When your application is slow, several questions may arise: Have I hit my bandwidth limit? Is my cluster under-provisioned and resource-constrained? Does my data model need to be optimized, or cause inefficient data access? Do my queries need to be more efficient, or are they missing necessary indexes? MongoDB Atlas provides tools to zoom in, uncover insights, and detect anomalies that might otherwise go unnoticed in vast data expanses. Let’s put it into practice Let's consider a hypothetical scenario to illustrate how to track down and address a performance bottleneck. Setting the context Imagine you run an online e-commerce store selling a popular item. On average, you sell about 500 units monthly. Your application comprises several services, including user management, product search, inventory management, shopping cart, order management, and payment processing. Recently, your store went viral online, driving significant traffic to your platform. This surge increased request latencies, and customers began reporting slow website performance. Identifying the bottleneck With multiple microservices, finding the service responsible for increased latencies can be challenging. Initial checks might show that inventory loads quickly, search results are prompt, and shopping cart updates are instantaneous. However, the issue might be more nuanced and time-sensitive, potentially leading to a full outage if left unaddressed. The five-step diagnostic process To resolve the issue, we’ll use a five-step diagnostic process: Gather data and insights by collecting relevant metrics and data. Generate hypotheses to formulate possible explanations for the problem. Prioritize hypotheses to use data to identify the most likely cause. Validate hypotheses by confirming or disproving the top hypothesis. Implement and observe to make changes and observe the results. Applying the five-step diagnostic process for resolution Let’s see how this diagnostic process unfolds: Step 1: Gather Data and Insights Customers report that the website is slow, so we start by checking for possible culprits. Inefficient queries, resource constraints, or network issues are the primary suspects. Step 2: Generate Hypotheses Given the context, the application could be making inefficient queries, the database could be resource-constrained, or network congestion could be causing delays. Step 3: Prioritize Hypotheses We begin by examining the Metric Charts in Atlas. Since our initial check revealed no obvious issues, we will investigate further. Step 4: Validate Hypotheses Using Atlas' Namespace Insights , we break down the host-level measurements to get collection-level data. We notice that the transactions.transactions collection has much higher latency than others. By increasing our lookback period to a week, the latency increased just over 24 hours ago when customers began reporting slow performance. Since this collection stores details about transactions, we use the Atlas Query Profiler to find that the queries are inefficient because they’re scanning through the whole transaction documents. This validates our hypothesis that application slowness was due to query inefficiency. Figure 1: New Query Insights Tab Step 5: Implement and Observe We need to create an index to resolve the collection scan issue. The Atlas Performance Advisor suggests an index on the customerID field. Adding this index enables the database to locate and retrieve transaction records for the specified customer more efficiently, reducing execution time. After creating the index, we return to our Namespace Insights page to observe the effect. We see that the latency on our transactions collection has decreased and stabilized. We can now follow up with our customers to update them on our fix and assure them that the problem has been resolved. Conclusion By gathering the correct data, working iteratively, and using the MongoDB observability suite , you can quickly resolve database bottlenecks and restore your application's performance. In our next post in the "Leveraging Database Observability at MongoDB" series, we’ll show how to integrate MongoDB metrics seamlessly into central observability stacks and workflows. This 'plug-and-play' experience aligns with popular monitoring systems like Datadog, New Relic, and Prometheus, offering a unified view of application performance and deep database insights in a comprehensive dashboard. Sign up for MongoDB Atlas , our cloud database service, to see database observability in action. For more information, see our Monitor Your Database Deployment docs page.
Inside MongoDB’s Early Access Programs
In tech, staying ahead of the curve is essential. Tech companies must continually innovate and release new developments to meet the ever-evolving needs of users and global market demands. One successful strategy is the use of Early Access Programs (EAPs) and Preview Features. These initiatives offer unique opportunities for both companies and users, fostering mutually beneficial relationships that drive product excellence. You may have heard them referred to as "Beta Programs," "Pilot Programs," or "Feature Previews," but they all fall under the same category of early user engagement aimed at refining products before general release. So what are Early Access Programs and Preview Features? EAPs and Preview Features allow a select group of users to test and provide feedback on new features before general release. These programs are aimed at loyal customers, power users, or those who’ve shown interest in the company's products. EAPs grant users early access to upcoming features or products still in development. Participants engage closely with the product team, providing valuable insights and feedback that shape the final product. Preview Features are specific features in a product made available for users to try before the official release. Unlike beta testing, which may involve a full product, preview features are often isolated components of a larger system. What are the Benefits of Early Access Programs? They offer numerous advantages for both companies and users. Direct feedback from real users helps identify bugs, usability issues, and areas for improvement that may not be evident during internal testing. This actionable feedback is crucial in enhancing product quality, allowing companies to release more polished and reliable features. This means higher user satisfaction and fewer post-release problems. EAPs also increase customer engagement and appreciation among participants by offering a hands-on exclusive experience to them. This can foster loyalty and strengthen the relationship between the company and its customers. These programs also provide market validation, offering an opportunity to gauge market interest and demand for new features. This enables companies to make data-driven decisions about which features to prioritize and further invest in. By identifying and resolving potential issues early in the development process, EAPs reduce the risk of major failures post-launch and allow for better resource allocation and planning. How does MongoDB approach Early Access Programs? At MongoDB, EAPs are classified into two categories: private preview and public preview. Private preview entails an invite-only white-glove experience for a select few participants who closely test and provide feedback. Public preview implies the feature is available to the public to try, either as downloadable tools or features in Atlas. Key Elements Selective Participation: MongoDB’s EAPs are typically invitation-only, targeting power users, those who’ve shown significant interest in new features, or are part of our MongoDB Champions community. This selective approach ensures that feedback comes from experienced users who can provide valuable insights. Direct Collaboration: Participants engage directly with MongoDB’s product and engineering teams. This direct line of communication allows for real-time feedback, in-depth discussions, and a collaborative approach to feature development. Structured Feedback Collection: We use a variety of methods to collect feedback, including surveys, structured interviews, and feedback forms. This structured approach ensures that feedback is actionable and can be effectively integrated into the development process. Iterative Development: Feedback is used to make iterative improvements to the features being tested. This agile approach allows us to quickly address issues, refine functionalities, and enhance the overall user experience. Transparency and Updates: We maintain open and transparent communication with our EAP participants, providing regular updates on the status of the features, changes made based on their feedback, and future development plans. This transparency fosters trust and keeps participants engaged throughout the program. Rewards and Recognition: Participants can share their stories on our social channels, be part of global events, and win MongoDB swag! Be Part of MongoDB’s Innovation MongoDB’s Early Access Programs offer participants the chance to gain early access to innovative features, influence product development, and join an exclusive, engaged community. As we continue to innovate and expand our product offerings, our Early Access Programs will help us deliver high-quality, user-centric solutions that empower our customers to achieve their strategic objectives. Join us in shaping the future of MongoDB by enrolling in our early access programs today!
magicpin Builds India's Largest Hyperlocal Retail Platform on MongoDB
Despite its trillion-dollar economy, 90% of retail consumption in India still takes place offline . While online retail in India has grown in recent years, much of it still consists of dark stores (a retail outlet or distribution center that exists exclusively for online shopping) and warehouses, the majority of retail establishments—fashion, food, dining, nightlife, and groceries—thrive as physical stores. What’s more, businesses looking to transition to online models are hindered by major platforms that focus primarily on clicks rather than encouraging transactions. This opportunity was the inspiration for the founders of magicpin , India’s largest hyperlocal retail platform. magicpin has revolutionized the conventional pay-per-click model, where businesses bid on keywords or phrases related to their products or services and then pay a fee each time someone clicks on an ad, with a new pay-per-conversion strategy. In a pay-per-conversion model, businesses only pay when they make an actual sale of a product or item. magicpin does not rely on dark stores, warehouses, or deep discounting; instead, it collaborates with local retailers, augmenting foot traffic and preserving the essence of local economies. This unique model ensures that consumers not only enjoy existing in-store benefits, but also receive additional perks when opting to transact through magicpin. “We enable the discovery of those merchants,” says Kunal Gupta, senior vice president at magicpin. “Which merchants in your local neighborhood are selling interesting stuff? What’s their inventory? What savings can we offer to buyers? We have data for everything.” Effectively three SaaS platforms in one, magicpin is a seller app, a buyer app, and a developing logistics app on the Open Network for Digital Commerce ( ONDC ), which is backed by the Indian government. With over 10 million users on its platform (covering the majority of Indian cities and over 100 localities), magicpin has established itself as a leading offline retail discovery and savings app. magicpin currently has 250,000 merchants in categories ranging from food to fashion to pharmacy. The power behind magicpin has always been MongoDB's flexibility and scalability. And from the company’s start in 2015, it became clear that magicpin was on to something special. “In the first week of March 2023 when we onboarded ONDC, we hit almost 10,000 transactions a day. In October last year, we peaked at 50,000 orders in a single day, which is a huge milestone,” says Kunal. “When an ONDC order is placed, it flows through us. We manage the entire process—from sending the order to the merchant, assigning logistics personnel for pickup and delivery, to handling any customer support tickets that may arise. It's the seamless integration of these elements that defines our contribution to the intricate framework of ONDC." Having launched using the community version of MongoDB , Kunal realized that magicpin needed to make better use of its relatively lean tech team and allow them to focus more on building the business. He also saw that a managed service would be a more effective way of handling maintenance and related tasks. “We realized there had to be a better solution. We can’t afford to have all the database expertise tied up with a team that’s focusing on creating businesses and building applications,” said Kunal. “That’s when we started to use MongoDB Atlas." magicpin uses a multitude of technologies, to store over 600 million SKUs, and handle its SaaS platform, session cache, card, and order management, and MongoDB Atlas sits at the heart of the business. “For our operational and scaling needs, it’s seamless,” Kunal concludes. “Availability is high, and monitoring and routing are super-good. Our lives have become much easier.” Watch the full presentation on YouTube to learn more.