daniel-rivers,-cto-&-co-founder,-pass-the-popcorn

2848 results

Unified Namespace Implementation with MongoDB and MaestroHub

In the complex world of modern manufacturing, a crucial challenge has long persisted: how to seamlessly connect the physical realm of industrial control systems with the digital landscape of enterprise operations. The International Society of Automation's ISA-95 standard, often visualized as the automation pyramid, has emerged as a guiding light. As shown below, this five-level hierarchical model empowers manufacturers to bridge the gap between these worlds, unlocking a path toward smarter, more integrated operations. Figure 1: In the automation pyramid, data moves up or down one layer at a time, using point-to-point connections. Manufacturing organizations face a number of challenges when implementing smart manufacturing applications due to the sheer volume and variety of data generated. An average factory produces terabytes of data daily, including time series data from machines stored in process historians and accessed by supervisory control and data acquisition (or SCADA) systems. Additionally, manufacturing execution systems (MES), enterprise resource planning (ERP) systems, and other operations software generate vast amounts of structured and unstructured data. Globally, the manufacturing industry generates an estimated 1.9 petabytes of data annually . Manufacturing leaders are eager to leverage their data for AI and generative AI projects, but a Workday Global Survey reveals that only 4% of the survey’s respondents believe their data is fully accessible for such applications. Data silos are a significant hurdle, with data workers spending an average of 48% of their time on data search and preparation. A popular approach to making data accessible is consolidating it in a cloud data warehouse and then adding context. However, this can be costly and inefficient, as dumping data without context makes it difficult for AI developers to understand its meaning and origin, especially for operational technology time series data. Figure 2: Pushing uncontextualized data to a data warehouse and then adding context is expensive and inefficient. All these issues underscore the need for a new approach—one that not only standardizes data across disparate shop floor systems, but also seamlessly weaves context into the fabric of this data. This is where the Unified Namespace (UNS) comes in. Figure 3: Unified Namespace provides the right data and context to all the applications connected to it. Unified Namespace is a centralized, real-time repository for all production data. It provides a single, comprehensive view of the business's current state. Using an event-driven architecture, applications publish real-time updates to a central message broker, which subscribers can consume asynchronously. This creates a flexible, decoupled ecosystem where applications can both produce and consume data as needed. Figure 4: UNS enables all the enterprise systems to have one centralized location to get the data they need for what they want to accomplish. MaestroHub and MongoDB: Solving the UNS challenge Initially introduced in 2011 at the Hannover Fair of Industrial Technologies, the core idea behind Industry 4.0 is to establish seamless connectivity and interoperability between disparate systems used in manufacturing. And UNS aims to solve this. Over the past five years, we have seen interest in UNS ramping up steadily, and now manufacturers are looking for practical ways to implement it. In particular, a question we’re frequently asked is where does UNS actually live. To answer that question, we need to look at popular architecture patterns, and the pros and cons of each. The most common pattern is implementing UNS in an MQTT broker. An MQTT broker will act as an intermediary entity that receives messages published by clients, filters the messages by topic, and distributes them to subscribers. The reason most manufacturers choose MQTT is it is an open architecture that is easy to implement. However, the challenge with just using the MQTT broker is that the clients don't get historical data access (which will be required to build the analytical and AI applications). Another approach can be to just dump all the data in a data warehouse and then add context to it. This can solve the problem of historical data access but it is an inefficient approach to standardize messages after they have been landed in the data warehouse in the cloud. A superior solution for comprehensive, real-time data access is combining a single source of truth (SSoT) Unified Namespace platform like MaestroHub with a flexible multi-cloud data platform like MongoDB Atlas. MaestroHub creates SSoT for industrial data, resulting in an up to 80% reduction in integration effort for brownfield facilities. Figure 5: MaestroHub SSoT creates a unified data integration layer, saving up to 50% of time in data contextualization (Source: MaestroHub). MaestroHub provides the connectivity layer to all data sources on the factory floor, along with contextualization and data orchestration. This makes it easy to connect the data needed for the UNS, enrich it with more context, and then publish it to consumers using the protocol that works best for them. Under the hood, MaestroHub stores metadata of connections, instances, and flows, and uses MongoDB as the database to store all this data. MongoDB’s flexible data modeling patterns reduce the complexity of mapping and transforming data when it's shared across different clients in the UNS. Additionally, scalable data indexing overcomes performance concerns as the UNS grows over time. Figure 6: MaestroHub and MongoDB together enable a real-time UNS plus long-term storage. MongoDB: The foundation for intelligent industrial UNS In the quest to build a unified namespace system (UNS) for the modern industrial landscape, the choice of database becomes paramount. So why turn to MongoDB? Scalability and high availability: It scales effortlessly, both vertically and horizontally (sharding), to handle the torrent of data from sensors, machines, and processes. Operational Technology (OT) systems generate vast amounts of data from these sources, and MongoDB ensures seamless management of that information. Document data model: Its adaptable document model accommodates diverse data structures, ensuring a harmonious flow of information. A Unified Namespace (UNS) must handle data from any factory source, accommodating structure variations. MongoDB's flexible schema design allows different data models to coexist in a single database, with schema extensibility at runtime. This flexibility facilitates the seamless integration of new data sources and types into the UNS. Real-time data processing: MongoDB Change Streams and Atlas Device Sync empower third-party applications to access real-time data updates. This is essential for monitoring, alerting, and real-time analysis within a UNS, enabling prompt responses to critical events. Gen AI application development with ease: Atlas Vector Search efficiently performs semantic searches on vector embeddings stored in MongoDB Atlas. This capability seamlessly integrates with large language models (LLMs) to provide relevant context in retrieval-augmented generation (RAG) systems. Given that the Universal Name Service (UNS) functions as a single source of truth for industrial applications, connecting gen AI apps to retrieve context from the UNS database ensures accurate and reliable information retrieval for these applications. With the foundational database established, let's explore MaestroHub, a platform designed to leverage the power of MongoDB in industrial settings. The MaestroHub platform MaestroHub is a provider of a SSoT for industrial data, specifically tailored for manufacturers. It achieves this through: Data connectors: MaestroHub connects to diverse data sources using 38 different industrial communication protocols, encompassing OT drivers, files, databases (SQL, NoSQL, time series), message brokers, web services, cloud systems, historians, and data warehouses. The bi-directional nature of 90% of these protocols ensures comprehensive data integration, leaving no data siloed. Data contextualization based on ISA-95: Leveraging ISA-95 Part 2, MaestroHub employs a semantic hierarchy and a clear naming convention for easy navigation and understanding of data topics. The contextualization of the payload is not just limited to the unit of measure AND definitional but also contains Enterprise/Site/Area/Line/Cell details, which are invaluable for analytics studies. Data contextualization is an important feature of a UNS platform. Logic flows/rule engine: Adhering to the UNS principle "Do not make any assumptions on how the data will be consumed," the data should flow flexibly from sources to brokers and from brokers to consumers in terms of rules, frequencies, and multiple endpoints. MaestroHub allows you to set rules such as Always, OnChange, OnTrue, and WhileTrue, where you can dynamically determine the conditions using events and inputs via JavaScript. Insights created by MaestroHub: MaestroHub provides real-time diagnostics of data health by leveraging Prometheus, Elasticsearch, Fluentd, and Kibana. Network problems, changed endpoints, and changed data types are automatically diagnosed and reported as insights. Additionally, MaestroHub uses NATS for queue management and stream analytics, buffering data in the event of a network outage. This allows IT and OT teams to monitor, debug, and audit logs with full data lineage. Conclusion The ISA-95 automation pyramid presents significant challenges for the manufacturing industry, including a lack of flexibility, limited scalability, and difficulty integrating new technologies. By adopting a Unified Namespace architecture with MaestroHub and MongoDB, manufacturers can overcome these challenges and achieve real-time visibility and control over their operations, leading to increased efficiency and improved business outcomes. Read more on how MongoDB enables Unified Namespace via its multi-cloud developer data platform. We are actively working with our clients on solving Unified Namespace challenges. Take a look at our Manufacturing and Industrial IoT page for more stories or contact us through the web form in the link.

June 18, 2024

Announcing MongoDB Server 8.0 Platform Support Improvements

Last month at MongoDB.local NYC 2024, we announced the preview of MDB 8.0 , the next evolution of MongoDB’s modern database. With MongoDB 8.0, we’re focused on delivering the unparalleled performance, scalability, and operational resilience necessary to support the creation of next-generation applications. For that to be possible, users must be able to deploy MongoDB on industry-standard operating systems. As a result, we are updating our Server Platform Policy to ensure that customers have the best possible experience when using MongoDB. Starting in MongoDB 8.0, there will be two new changes: When a new major version of MongoDB is released, we will only release it operating system (OS) versions that are fully supported by the vendor for the duration of the MongoDB version’s life. In short, we will support an operating system if the operating system’s Extended Lifecycle Support (ELS) date is after the MongoDB Server’s End of Life (EOL) date. We will release new MongoDB Server versions (both major and minor) on the minimum supported minor version of the OS (defined by the OS vendor). Once an OS minor version is no longer supported by the vendor, we will update future MongoDB Server versions to the next supported OS minor version. As always, MongoDB reserves the right to discontinue support for platforms based on lack of user demand and/or technical difficulties (e.g., if a platform doesn’t support required libraries or compiler features). Ensuring best-in-class security MongoDB routinely updates our documentation to indicate which platforms a new version of the MongoDB Server will be available on with the general availability release of that new server version. To ensure that MongoDB customers can meet strong regulatory and security requirements, our software is developed, released, and distributed in accordance with industry security best practices. Given the mission-critical nature of MongoDB’s business—providing a highly secure, performant data platform to tens of thousands of customers in over 100 countries—we strive to provide strong and consistent security assurances across all of our products. In addition, MongoDB partners also need guarantees about the security development lifecycle of our products so they can provide the best experience to their customers. By ensuring that our software runs only on platform versions that are receiving security patches, we aim to limit the vulnerabilities that might be introduced by customers running EOL operating systems. The significance of this change With every major server release, MongoDB determines the supported builds for that general availability (GA) release according to the planned vendor platform’s end of life date —meaning the MongoDB major release will not support the operating system if the operating system’s extended lifecycle support ends before the MongoDB EOL date. This also applies to server container images delivered to our customers. Furthermore, to guarantee security assurances for operating systems that have a minimum minor version, we will only build new versions of MongoDB Server software on a vendor-supported major/minor version of the operating system. Concretely, we will build new versions of MongoDB on a minimum minor version until it hits a maintenance event (defined on a per-vendor basis), and at that point future MongoDB server builds will be updated to the new supported minor version. Separately, when a vendor publishes a new major version of an operating system after a given version of MongoDB reaches GA, we will evaluate whether the latest MongoDB release will run on this new OS version, or we will wait for the next major MongoDB release before documenting formal platform support on our website. Walkthrough: How it could work for you Consider the RHEL 9 Planning Guide below and the hypothetical release cadence of MongoDB version X.0. As long as version X.0 is released three years before the end of RHEL 9 support, which as noted by RHEL is 2032 , we will provide support on RHEL 9. This means that 2029 will be the last year that MongoDB releases a server version on RHEL 9. Next, consider that version X.0 will be released at the end of 2025. Following the Extended Update Support Plan, we will build version X.0 on RHEL 9.6 until the start of 2026 when RHEL 9.8 becomes available. And then for future versions, MDB X.Y will begin being built on RHEL 9.8 until we require the minimum version to be 9.10 in 2027. RHEL 9 planning guide Building the future Overall, these coming changes to the MongoDB Server Platform Policy underscore MongoDB’s commitment to helping developers innovate quickly and easily while providing an even more highly secure and performant data platform. Stay tuned for additional updates about MongoDB 8.0—which will provide optimal performance by dramatically increasing query performance, improving resilience during periods of heavy load, making scalability easier and more cost-effective, and making time series collections faster and more efficient. For more information about the Server Platform Policy updates, please refer to our documentation .

June 17, 2024

AI-Powered Media Personalization: MongoDB and Vector Search

In recent years, the media industry has grappled with a range of serious challenges, from adapting to digital platforms and on-demand consumption, to monetizing digital content, and competing with tech giants and new media upstarts. Economic pressures from declining sources of revenue like advertising, trust issues due to misinformation, and the difficulty of navigating regulatory environments have added to the complexities facing the industry. Additionally, keeping pace with technological advancements, ensuring cybersecurity, engaging audiences with personalized and interactive content, and addressing globalization issues all require significant innovation and investment to maintain content quality and relevance. In particular, a surge in digital content has saturated the media market, making it increasingly difficult to capture and retain audience attention. Furthermore, a decline in referral traffic—primarily from social media platforms and search engines—has put significant pressure on traditional media outlets. An industry survey from a sample of more than 300 digital leaders from more than 50 countries and territories shows that traffic to news sites from Facebook fell 48% in 2023, with traffic from X/Twitter declining by 27%. As a result, publishers are seeking ways to stabilize their user bases and to enhance engagement sustainably, with 77% looking to invest more in direct channels to deal with the loss of referrals. Enter artificial intelligence: generative AI-powered personalization has become a critical tool for driving the future of media channels. The approach we discuss here offers a roadmap for publishers navigating the shifting dynamics of news consumption and user engagement. Indeed, using AI for backend news automation ( 56% ) is considered the most important use of the technology by publishers. In this post, we’ll walk you through using MongoDB Atlas and Atlas Vector Search to transform how content is delivered to users. The shift in news consumption Today's audiences rarely rely on a single news source. Instead, they use multiple platforms to stay informed, a trend that's been driven by the rise of social media, video-based news formats, and skepticism towards traditional media due to the prevalence (or fear) of "fake news." This diversification in news sources presents a dilemma for publishers, who have come to depend on traffic from social media platforms like Facebook and Twitter. However, both platforms have started to deprioritize news content in favor of posts from individual creators and non-news content, leading to a sharp decline in media referrals. The key to retaining audiences lies in making content personalized and engaging. AI-powered personalization and recommendation systems are essential tools for achieving this. Content suggestions and personalization By drawing on user data, behavior analytics, and the multi-dimensional vectorization of media content, MongoDB Atlas and Atlas Vector Search can be applied to multiple AI use cases to revolutionize media channels and improve end-user experiences. By doing so, media organizations can suggest content that aligns more closely with individual preferences and past interactions. This not only enhances user engagement but also increases the likelihood of converting free users into paying subscribers. The essence of leveraging Atlas and Vector Search is to understand the user. By analyzing interactions and consumption patterns, the solution not only grasps what content resonates but also predicts what users are likely to engage with in the future. This insight allows for crafting a highly personalized content journey. The below image shows a reference architecture highlighting where MongoDB can be leveraged to achieve AI-powered personalization. To achieve this, you can integrate several advanced capabilities: Content suggestions and personalization: The solution can suggest content that aligns with individual preferences and past interactions. This not only enhances user engagement but also increases the likelihood of converting free users into paying subscribers. By integrating MongoDB's vector search to perform k-nearest neighbor (k-NN) searches , you can streamline and optimize how content is matched. Vectors are embedded directly in MongoDB documents, which has several advantages. For instance: No complexities of a polyglot persistence architecture. No need to extract, transform, and load (ETL) data between different database systems, which simplifies the data architecture and reduces overhead. MongoDB’s built-in scalability and resilience can support vector search operations more reliably. Organizations can scale their operations vertically or horizontally, even choosing to scale search nodes independently from operational database nodes, flexibly adapting to the specific load scenario. Content summarization and reformatting: In an age of information overload, this solution provides concise summaries and adapts content formats based on user preferences and device specifications. This tailored approach addresses the diverse consumption habits of users across different platforms. Keyword extraction: Essential information is drawn from content through advanced keyword extraction, enabling users to grasp key news dimensions quickly and enhancing the searchability of content within the platform. Keywords are fundamental to how content is indexed and found in search engines, and they significantly influence the SEO (search engine optimization) performance of digital content. In traditional publishing workflows, selecting these keywords can be a highly manual and labor-intensive task, requiring content creators to identify and incorporate relevant keywords meticulously. This process is not only time-consuming but also prone to human error, with significant keywords often overlooked or underutilized, which can diminish the content's visibility and engagement. With the help of the underlying LLM, the solution extracts keywords automatically and with high sophistication. Automatic creation of Insights and dossiers: The solution can automatically generate comprehensive insights and dossiers from multiple articles. This feature is particularly valuable for users interested in deep dives into specific topics or events, providing them with a rich, contextual experience. This capability leverages the power of one or more Large Language Models (LLMs) to generate natural language output, enhancing the richness and accessibility of information derived from across multiple source articles. This process is agnostic to the specific LLMs used, providing flexibility and adaptability to integrate with any leading language model that fits the publisher's requirements. Whether the publisher chooses to employ more widely recognized models (like OpenAI's GPT series) or other emerging technologies, our solution seamlessly incorporates these tools to synthesize and summarize vast amounts of data. Here’s a deeper look at how this works: Integration with multiple sources: The system pulls content from a variety of articles and data sources, retrieved with MongoDB Atlas Vector Search. Found items are then compiled into dossiers, which provide users with a detailed and contextual exploration of topics, curated to offer a narrative or analytical perspective that adds value beyond the original content. Customizable output: The output is highly customizable. Publishers can set parameters based on their audience’s preferences or specific project requirements. This includes adjusting the level of detail, the use of technical versus layman terms, and the inclusion of multimedia elements to complement the text. This feature significantly enhances user engagement by delivering highly personalized and context-rich content. It caters to users looking for quick summaries as well as those seeking in-depth analyses, thereby broadening the appeal of the platform and encouraging deeper interaction with the content. By using LLMs to automate these processes, publishers can maintain a high level of productivity and innovation in content creation, ensuring they remain at the cutting edge of media delivery. Future directions As media consumption habits continue to evolve, AI-powered personalization stands out as a vital tool for publishers. By using AI to deliver tailored content and to automate back end processes, publishers can address the decline in traditional referrals and build stronger, more direct relationships with their audiences. If you would like to learn more about AI-Powered Media Personalization, visit the following resources: AI-Powered Personalization to Drive Next-Generation Media Channels AI-Powered Innovation in Telecommunications and Media GitHub Repository : Create a local version of this solution by following the instructions in the repository

June 13, 2024

Helping MongoDB Customers Unlock Potential with Industry Solutions

Gabriela Preiss is a senior manager within Industry Solutions at MongoDB and was instrumental in building out the team in Barcelona. She’s now relocated to Austin, Texas to build a team in Mexico City and continue expanding the Industry Solutions footprint. In this article, Gabriela shares more about Industry Solutions at MongoDB and how they’re making a difference for both customers and our internal go-to-market teams. The impact of industry solutions teams is not lost on the tech industry. I see our competitors and Big Tech alike becoming more verticalized and providing industry-specific solutions to meet their customers’ needs. In 2019, MongoDB established an Industry Solutions team to understand and address our customers’ industry-specific needs and challenges. In the past two years, our Industry Solutions team has increased by roughly 380% in size and touched over 1,100 customer accounts around the globe. Our tailored, industry-specific solutions and messaging give MongoDB a competitive advantage and lead to higher market penetration, higher customer retention, and increased sales. Not to mention, industry-specific insights gleaned from the field drive internal innovation and product development to propel MongoDB forward. Industry Solutions at MongoDB The Industry Solutions team messages MongoDB as a solution or part of a solution to specific industries. We speak the customer's language and understand their industry needs, industry roadblocks, market trends, and competitors. Our industry experts have been in the shoes of the customers and know how to guide them through modernization. We're an extremely cross-functional team, constantly collaborating with sales, marketing, product, product marketing, and engineering. We work in parallel with sales, and everything we do is ultimately to support them and help drive revenue. This means helping with account prepping, speaking one-on-one with customers, and a lot of sales enablement. Our team holds frequent sales trainings in the form of internal content, office hours, and weekly sessions to coach on industry knowledge. We also create content to help drive MongoDB’s go-to-market messaging externally. This means highlighting MongoDB as an industry solution through blogs, white papers, video content, and, most frequently, interactive solution demos that allow customers to really get their hands on our products. Sharing industry knowledge at MongoDB.local events Skills for success While you don’t necessarily need to be a MongoDB expert to join our Industry Solutions team (we’ll train you on that); it’s beneficial to have foundational technical skills and knowledge, like understanding the concept of a database and how it works. As long as you have the will to learn, we’ll shape you into a MongoDB subject matter expert. I often look for people who have a technical knowledge base, but are also interested in the business solution space. Our team is open for anyone who wants to be part of industry solutions and has the willingness to learn all of the ins and outs of it. In terms of specific skills, I personally think soft skills are the most essential for success. For example, having a willingness to learn, boundless curiosity, a sense of urgency, and a true passion for your work. However, things like strong project management and time management skills come in handy as our team covers many different industries and regions. Each industry has its own nuances, and each team member is involved in many different projects. The ability to own your projects from end-to-end and juggle multiple projects at any given time is crucial. Working with AI AI has become a hot topic, and it's not going away. As a team, we work with industries from financial services to manufacturing, automotive to airline, insurance to telecommunications, and everything in between. AI is affecting every industry, and it's impacting every region. It’s our job to keep our finger on the pulse of industry trends to better enable our sales team and have discussions of modernization with customers. As customers look to implement gen AI applications, it’s important for our team to be able to confidently answer their questions and create relevant content. For the Industry Solutions team, working with gen AI is all about educating ourselves and keeping up with the industry trends to create a competitive advantage for MongoDB. Collaboration never ends! Opportunities for learning and growth There are so many, I wouldn't even know where to begin or end. As far as opportunities for learning, you’ll certainly become a subject matter expert in MongoDB. You’ll have the chance to speak with different customers, participate as a presenter at MongoDB and third-party events, hold internal sales enablement trainings, hone in on your content creation skills, and more. You’ll learn some teaching skills if you join the Industry Solutions team, too. A big part of the role is explaining technical concepts to different audiences and sharing information in a way that enables people to learn. At the end of the day, you’ll be building your brand as an industry expert, building your skills, and building your resume. In my personal experience, I started as a consultant on the team and am now a senior manager. I was able to build out our team in Barcelona, Spain, and I've recently relocated to Austin, Texas to build out a team in Mexico City, which I'm so excited for. It's been amazing to see the team grow. In terms of career advancement, there are tons of opportunities for our team members to grow linearly as individual contributors or into team lead and people manager roles. Plus, because you're getting exposure to so many different teams, it's not uncommon for folks from Industry Solutions to transfer into other departments within MongoDB, or vice versa. It’s something that’s not taboo within the company; it's just a matter of having a conversation with your manager. Rooftop views from MongoDB Barcelona Building a team in Mexico City and beyond It’s a really exciting time as we aim to replicate the team that we've created in Barcelona in Mexico City. For prospective candidates, this is a great opportunity to help us build and shape a team from scratch while working on a diverse set of projects in key regions of the business. I would love to see the Mexico City office grow in the same way our Barcelona office has, which, a few years ago was only 12 people and now contains hundreds. We’ll continue to hire for our team in Barcelona and other locations around the world, too. I’m looking forward to bringing on people who will add unique perspectives to our team and grow their careers at MongoDB for years to come. Join our talent community to stay up to date with MongoDB culture content and career opportunities.

June 12, 2024

Building Gen AI with MongoDB & AI Partners: May 2024

Since I joined MongoDB last September, each month has seemed more action-packed than the last. But it’s possible that May was the busiest of all: May 2024 was a month of big milestones for MongoDB! First, we held MongoDB.local NYC on May 2, our biggest .local event so far, with 2,500 attendees from around the world. It was the first MongoDB.local event I attended since joining the company, and suffice it to say I was thrilled to meet with so many colleagues and partners in person. I was particularly excited to discuss the impact of MongoDB Atlas on the generative AI space, since we also announced the new MongoDB AI Applications Program (MAAP) in May. MongoDB’s CEO, Dev Ittycheria, on the MongoDB .Local NYC keynote stage MAAP was launched to help organizations quickly build, integrate, and deploy gen AI-enriched applications at scale. We do this by providing customers a complete package that includes strategic advisory, professional services, and a robust tech stack through MongoDB and our amazing partners: Anthropic, Anyscale, Amazon Web Services (AWS), Cohere, Credal.ai, Fireworks.ai, Google Cloud, gravity9, LangChain, LlamaIndex, Microsoft Azure, Nomic, PeerIslands, Pureinsights, and Together AI. I really look forward to seeing how MAAP will empower customers to create secure, reliable, and high-performing gen AI applications after the program becomes publicly available in July. Stay tuned for more! And if you’re interested in hearing more about MongoDB’s approach to AI partnerships, and how MAAP will help organizations of all sizes build gen AI applications, check out my interview with theCUBE at MongoDB.local NYC alongside Benny Chen, co-founder of Fireworks.ai. Upcoming AI partner events Are you in San Francisco in late June? We’re proud to sponsor the AI Engineer World’s Fair this year! Stop by the MongoDB booth to chat about gen AI development, and make sure to attend our panel “Building Your AI Stack with MongoDB, Cohere, LlamaIndex, and Together AI” on June 27. Welcoming new AI partners In addition to .local NYC and announcing MAAP in May, we also welcomed four AI partners that offer product integrations with MongoDB: Haystack, Mixpeek, Quotient AI, and Radiant. Read on to learn more about each great new partner. Haystack is an open source Python framework for building custom apps with large language models (LLMs). It allows users to try out the latest models in natural language processing (NLP) while being flexible and easy to use. “We’re excited to partner with MongoDB to help developers build top-tier LLM applications,” said Malte Pietsch, co-founder and CTO of deepset , makers of Haystack and deepset Cloud. “The new Haystack and MongoDB Atlas integration lets developers seamlessly use MongoDB data in Haystack, a reliable framework for creating quality LLM pipelines for use cases like RAG, QA, and agentic pipelines. Whether you're an experienced developer or just starting, your gen AI projects can quickly progress from prototype to adoption, accelerating value for your business and end-users." Learn more about Haystack’s MongoDBAtlasDocumentStore to improve your AI applications. Mixpeek is a multimodal indexing pipeline that gets a database ready for generative AI. It allows developers to treat an object store and a transactional database as a single entity. Ethan Steininger, founder of Mixpeek, explained the value of the MongoDB-Mixpeek integration. “With MongoDB, developers store vectors, metadata, text and all the indexes needed for hyper-targeted retrieval,” he said. “Combined with Mixpeek, they can ensure their S3 buckets and all the documents, images, video, audio and text objects are always consistent with their transactional database, accelerating the path to production by instilling confidence that multimodal RAG results will always be up-to-date." Read more about our partnership and learn how to build real-time multimodal vectors in a MongoDB cluster. Quotient AI is a solution that offers developers the capability to evaluate their AI products with specialized datasets and frameworks to accelerate the experimentation cycle. Julia Neagu, CEO of Quotient AI, highlighted the importance of our partnership. "We are excited to join forces with MongoDB and revolutionize how developers and enterprises are building AI products,” she said. “We share the common goal of helping developers get their ideas to market faster with a first-class developer experience. MongoDB Atlas scalable and versatile vector database technology complements Quotient's mission to ship high-quality, reliable AI applications through rapid, domain-specific evaluation." Learn more how Quotient AI enables evaluation and refinement of RAG-powered AI products built on MongoDB Atlas. Radiant offers a monitoring and evaluation framework for production AI use cases. Nitish Kulnani, CEO of Radiant, shared his excitement about the partnership with MongoDB to enhance the reliability of AI applications. “By combining Radiant's anomaly detection with MongoDB Atlas Vector Search, we enable developers to swiftly identify and mitigate risks, and quickly deploy high-quality AI solutions, delivering real value to customers faster,” he said. “MongoDB trusts Radiant to accelerate its own AI applications, and we're excited to deliver the same experience to MongoDB customers.'' Read more about how to deploy Radiant with MongoDB Atlas to accelerate your journey from development to production. But wait, there’s more! To learn more about building AI-powered apps with MongoDB, check out our AI Resources Hub , and stop by our Partner Ecosystem Catalog to read about our integrations with MongoDB’s ever-evolving AI partner ecosystem.

June 5, 2024

Spinning Up Innovation: How MongoDB is Driving New Solutions at QSSTUDIO

Building bespoke innovations that address the needs of the future is the driving force behind the unique tech consultancy QSSTUDIO. With global venture capital funding forecast by EY to reach $12 billion this year for generative AI startups alone, the ability to bring ideas to life requires the right combination of technology and understanding of the app creation process. Led by Robin Tang, QSSTUDIO is a bespoke app and web platform development consultancy that provides professional digital expertise to dozens of small businesses, digital agencies, and startups in Australia and abroad. The flexibility of MongoDB Atlas, an integrated suite of data services, gives QSSTUDIO an instant launch pad for new ideas, allowing the team to easily spin-up new solutions and prototypes for clients. From skincare AI, to mining safety tools, and the petrol price tracking app ServoTrack, the team’s technological expertise combines with an understanding of data to bring these new ideas to market. QSSTUDIO has been able to keep its development team small, in part by using MongoDB Atlas Cluster, a NoSQL Database-as-a-Service in the cloud, which enables the team to easily test proof-of-concept designs and then migrate successful apps to Atlas without the need to reprovision the product. “We build bespoke solutions, and the flexibility and scalability of MongoDB Atlas has made it so much easier for us to quickly innovate and iterate, from concept inception to deployment,” Tang says. Tracking the ups and downs of Australia’s petrol prices With petrol prices soaring past AU$2.50/L, the ServoTrack app, founded by Robin Tang, is one of the best ways Australians can look for lower prices as they fluctuate during the week. Almost all of Australia’s state governments now supply unstructured data sources of real-time petrol price information. Tang wanted to create an app that allowed regular people to set alerts and be notified in real time of rises and falls in price. MongoDB Atlas provides the perfect solution to broadcast the data from these reputable sources. “The flexible data structure of MongoDB allows us to use different data schemas for data from different states. With geospatial indexing as well as Atlas Search Indexing, it provides a speedy way to allow users to find their target service station,” Tang says. “Atlas and MongoDB App services provide a few additional benefits for ServoTrack,” he added. “As expected for any cloud provider, the ability to scale whilst retaining automated backups means I could develop without worrying too much about our infrastructure.” Tang can leverage built-in features such as authentication, which allows users to sign in and store app preferences no matter how many devices they owned, whether it is on Android or iOS, as well as provide updates to users. “We use MongoDB Atlas to enable triggers on price changes at set service stations or general locations depending on the data available. This has helped to alert thousands of users of pending price increases, allowing them to save hundreds of dollars per year,” Tang says. RockShield: an innovative solution for miner safety QSSTUDIO created RockSHIELD software that has gone into the smart geotechnical instrumentation solutions delivered by SCT Operations. Recently, the underground mining and civil tunneling industry has undergone a digital transformation, with geotechnical instrumentation evolving from analogue to digital, resulting in more accurate field data that is accessed more frequently. Mining is an inherently high-risk operation, so it was essential to use this improved monitoring data to detect rock movement early and maintain a safer working environment for miners. Using MongoDB, Tang and his team created RockSHIELD’s software, which uses sensors to maintain mine safety by detecting the slightest movement in rocks. “We took the data on displacement, rotation, and angles from the rock sensors and put it into the MongoDB database which can handle flexible data sets,” Tang said. “Using MongoDB, we have the flexibility to store and receive data from a variety of sensors in different formats. The moment the numbers hit a threshold, the dashboard receives a notification that there is an issue. “The clients adopting this solution are very security conscious, and MongoDB’s security features make it attractive for our clients,” Tang says. “Our use of x.509 certificates for authentication, TLS/SSL communication, and Role Based Access Control let us create a hierarchy of access suitable for SCT Operations as well as RockSHIELD’s clients.” It was important to the team to create a system that works and is sustainable in the long-term to improve and maintain as high safety standards as possible. If you want to level-up your organization's innovation, or you’re a startup looking to build proof of concept, it's easy to get started with MongoDB Atlas today.

June 4, 2024

Microservices: Realizing the Benefits Without the Complexity

The microservice architecture has emerged as the preferred, modern approach for developers to build and deploy applications on the cloud. It can help you deliver more reliable applications, and address the scale and latency concerns for System Reliability Engineers (SREs) and operations. But microservices aren't without their hangups. For developers, microservices can lead to additional complexity and cognitive overhead, such as cross-service coordination, shared states across multiple services, and coding and testing failure logic across disconnected services. While the monolith was suboptimal for compute and scale efficiencies, the programming model was simple. So the question is, can we get the best of both worlds? In addition, how do we make the individual services easier to build and adapt to changing requirements? Since, at their core, microservices provide access to and perform operations on data, how do we architect services so that developers can easily work with data? How can we make it easier for developers to add new types of data and data sources and perform a wide variety of data operations without the complexity of managing caches and using multiple query languages (SQL, full-text and vector search, time series, geospatial, etc.) The development complexity associated with microservice architectures occurs at two levels: service orchestration and service data management. The diagram below depicts this complexity. At the orchestration level, a typical application may support tens or hundreds of processes, and each may have thousands or millions of executions. To make this work, services are often connected by a patchwork of queues. Developers spend quite a bit of time tracking and managing all the various workflows. The sheer scale necessitates a central mechanism to manage concurrent tasks and sharded databases to manage the state of millions of concurrent workflow instances. To add more complexity, each microservice is deployed using a set of data platforms including RDBMS, caches, search engines, and vector and NoSQL databases. Developers must work with multiple query languages, write code to keep data in sync among these platforms and write code to deal with edge cases when invariably data or indexes are not in sync. Finally, developer productivity is inhibited by the brittleness of RDBMS, which lacks flexibility when trying to incorporate new or changing data types. As a result, microservice applications often end up with complex architectures that are difficult to develop against and maintain in terms of both the individual microservices and the service orchestration. Realizing the benefits without the complexity One approach to address these microservice challenges is to combine two technologies: Temporal and MongoDB. Both give you the benefits of microservices while simplifying the implementation of service orchestration. Together, they allow developers to build services that can easily handle a wide variety of data, eliminate the need to code for complex infrastructure and reduce the likelihood of failure. They simplify the data model and your code. In one real-world example, open-source indexing company Mixpeek leverages the combination of MongoDB and Temporal to provide a platform enabling organizations to easily incorporate multi-modal data sources in AI applications. Mixpeek’s CEO Ethan Steininger states, “Temporal’s durable execution guarantees and MongoDB's flexible data model are core components of Mixpeek’s multimodal data processing and storage. Combined, they enable our users to run high volume ML on commodity hardware without worrying about dropped jobs.” MongoDB and Temporal: Build like a monolith with durable microservices Both MongoDB and Temporal were built by developers, for developers. They both use a code-first approach to solving the complex infrastructure needs of our modern applications within our application code and empower developers to be more productive. They are part of an emerging development stack that greatly simplifies data and all the cross-functional coordination we need in our cloud applications. Ultimately, the combination of these two developer-focused platforms allows you to simplify design, development, and testing of microservice-based applications. With the document model of MongoDB, you model data as real world objects and not tables, rows, and columns. With Temporal, you design your end-to-end service flows as workflows as described by domain experts without having to explicitly identify every edge case and exception (Temporal handles those implicitly). Temporal and MongoDB provide the same benefits that, when combined, multiply in value. You become more agile, as not only can everyone understand your code better, but you are no longer challenged by the cognitive overload of trying to coordinate, comprehend, and test a web of disconnected and complex services. Together, they allow us to reliably orchestrate business processes within apps that are all speaking the language of the data itself. Combining Temporal and MongoDB results in the simplified architecture shown below. Temporal enables orchestration to be implemented at a higher level of abstraction, eliminating much of the event management and queuing complexity. MongoDB, in turn, provides a single integrated data platform with a unified query language thereby eliminating much of the data management complexity. Let’s examine MongoDB and Temporal in more depth to better understand their capabilities and why they facilitate the rapid development of microservices-based applications. MongoDB: Simplifying microservice data MongoDB's features align well with the principles of microservices architectures. It reduces the need for niche databases and the associated costs of deploying and maintaining a complicated sprawl of data technologies. More explicitly, MongoDB delivers key benefits for microservice development: Flexible schema, flexible services: Unlike relational databases with rigid schemas, MongoDB's document model allows microservices to easily evolve as data requirements change. Distributed scale for data-heavy, distributed services: MongoDB scales horizontally by adding more partitions to distribute the load. This aligns with the modular nature of microservices, where individual services can scale based on their specific needs. Unified query language reduces microservice sprawl: MongoDB supports a diverse set of data operations without requiring multiple data platforms (caches, vector, and text search engines, time series, geospatial, etc.) Operational efficiency: MongoDB Atlas, the cloud-based version of MongoDB, simplifies managing databases for microservices. It handles provisioning, backups, and patching, freeing developers to focus on core responsibilities. Integrated developer data platform: The integrated developer data platform delivers an intuitive set of tools to build services that support mobile clients, real-time analytics, data visualization, and historical analysis across many service databases. With MongoDB, development teams use one interface for all their services and run it anywhere, even across clouds. Also, it provides a data foundation for your microservices that is highly available, scalable, and secure. It greatly simplifies microservices development so that you can focus on your business problems and not data. Temporal: Don't coordinate services, orchestrate them Temporal delivers an open-source, durable execution solution that removes the complexity of building scalable distributed microservices. It presents a development abstraction that preserves the complete application state so that in the case of a host or software failure, it can seamlessly migrate execution to another machine. This means you can develop applications as if failures—like network outages or server crashes—do not exist. Temporal handles these issues, allowing you to focus on implementing business logic rather than coding complex failure detection and recovery routines. Here's how Temporal simplifies application development: Durable workflows: Temporal maintains the state and progress of a defined workflow across multiple services, even in the face of server crashes, network partitions, and other types of failures. This durability ensures that your application logic can resume where it left off, making your overall application more resilient. Simplifies failure handling: Temporal abstracts away the complex error handling and retry logic that developers typically have to implement in distributed systems. This abstraction allows developers to focus on business logic rather than the intricacies of ensuring their end-to-end services can handle failures gracefully. Scale: Temporal applications are inherently scalable and capable of handling billions of workflow executions. Long-running services: Temporal supports long-running operations, from seconds to years, with the same level of reliability and scalability. By providing a platform that handles the complexities of distributed systems, Temporal allows developers to concentrate on implementing business logic in their services. This focus can lead to faster development times and more reliable applications, as developers are not bogged down by the intricacies of state management, retries, and error handling. The next generation of microservices development is here Developers want to code. They want to solve business problems. They do not want to be bogged down by the complexity of infrastructure failures. They want to model their apps and data so that it is aligned with the real-world entities and domains they are solving for. Using MongoDB and Temporal together solves these complexities. Together, they simplify design, development, and testing of microservice-based applications so that you can focus on business problems and deliver more features faster. Getting started with Temporal and MongoDB Atlas We can help you design the best architecture for your organization’s needs. Feel free to connect with your MongoDB and Temporal account teams or contact us to schedule a collaborative session and explore how Temporal and MongoDB can optimize your AI development process.

June 3, 2024

AWS Names MongoDB ASEAN Global Software Partner of the Year

I’m thrilled to announce that during the recent AWS Partner Summit in Bangkok , MongoDB was recognized as the ASEAN Global Software Partner of the Year — for the second year in a row. This award highlights MongoDB's focus on driving innovation with AWS, and is a testament to the success of customers in the region building transformative, next-generation applications with MongoDB and AWS. Based on merit, the AWS ASEAN Partner Awards were determined through a data-driven decision-making process, and “celebrated stellar achievements from partners that have shown remarkable success and achievement with AWS.” “We are proud to announce and acknowledge the ASEAN AWS Partners of 2024 that are helping customers accelerate innovation, develop industry-focused solutions, and build resilience amid the current evolving economic climate,” said Kirsten Gilbertson, Partner Organization Leader, AWS ASEAN. “AWS Partners are the force multiplier to accelerate cloud transformation in the region and drive local economic growth. With our partners, AWS remains committed to helping customers address industry needs and drive positive business and societal outcomes by leveraging our secure global infrastructure for the latest generative—AI, analytics, and machine learning technologies.” Demystifying AI and building next-gen apps with partners To help organizations of all sizes make the most of AI, earlier this month, we announced the MongoDB AI Applications Program (MAAP). The new program is designed to help organizations rapidly build and deploy modern applications with generative AI technology at enterprise scale. MAAP—which will be available starting in July—is being launched with industry-leading consultancies, cloud infrastructure, and generative AI framework providers, including AWS. Together, MongoDB and its MAAP partners will help customers solve business problems with AI by providing them with strategic advisory, professional services, and an integrated end-to-end technology stack. In short, MongoDB and partners like AWS, Anthropic, and Cohere will help customers use generative AI to enhance productivity, revolutionize customer interactions, and drive industry advancements. So it’s a real honor to receive the AWS ASEAN Global Software Partner of the Year two years running. This award validates the strength of our strategic collaboration with AWS, and the growing number of customers across industries who deploy mission-critical workloads on MongoDB Atlas running on AWS. We look forward to building on this momentum with AWS to help our customers unlock new possibilities by leveraging the power of data! To learn more about the MongoDB AI Applications Program, visit the MAAP page . And, check out MongoDB’s Partner Ecosystem to learn more about the wide range of integrations and solutions to help you build and run modern applications.

June 3, 2024

Leveraging Database Observability at MongoDB: Unlocking Performance Insights and Optimization Strategies

This post is the first in a three-part series on leveraging database observability. Observability has evolved into an essential information technology component, offering advanced insights into system performance beyond traditional monitoring. While monitoring aims to identify problems, observability helps understand and resolve them. Businesses prioritizing observability experience less downtime, leading to enhanced user experiences and improved ROI. Indeed, Splunk’s The State of Observability 2023 report quantified the financial impact of downtime—more than $150,000 per hour. Furthermore, observability leaders reported 33% fewer outages and achieved eight times better ROI than new adopters. Throughout this series, we'll define database observability at MongoDB, explore our suite of tools, delve into third-party integrations, and discuss everyday use cases. We will also establish a shared methodology and vocabulary for discussing observability at MongoDB and highlight the tools and features that have delivered value for our customers. Observability and MongoDB’s strategy Monitoring involves using tools to track real-time operations and alert teams to issues. As defined by Gartner, observability evolves monitoring into a process that provides deep insights into digital business applications, enhancing innovation and customer experience. The key difference is that monitoring detects the presence of issues, while observability gathers detailed information to understand and resolve them, which is crucial for modern IT infrastructure needs. Databases, in particular, play a critical role in this IT ecosystem, where performance and resilience directly impact business outcomes. This advancement is essential for DevOps, Database Administrators, and economic buyers responsible for these databases, as it enhances system reliability, encourages innovation, and supports financial objectives. Ultimately, observability provides comprehensive insights into system performance, health, and reliability by seamlessly integrating and contextualizing telemetry data. MongoDB leverages a unique observability strategy with out-of-the-box tools that automatically monitor and optimize customer databases. Explicitly designed for MongoDB environments, our system provides continuous feedback and answers critical questions—What is happening? Where is the issue? Why is it occurring? How do I fix it?—to enhance performance, increase productivity, and minimize downtime. Supporting MongoDB Atlas (our fully managed platform), Cloud Manager, and Ops Manager, as well as tailored monitoring solutions for the full range of developer data platform products (from enhanced search functionalities to app services and search nodes). Our approach meets the evolving needs of customer applications by: Leveraging MongoDB expertise: The MongoDB observability suite integrates efficiency and best practices from the beginning of the development cycle. As MongoDB platform experts, we use our deep knowledge to provide top-tier optimization insights. We apply our extensive understanding of our tools to ensure our customers benefit from a high-performing and resilient database. Offering streamlined metrics: We integrate our metrics seamlessly into our customers' central observability stacks and workflows. This creates a 'plug-and-play' experience that effortlessly aligns with popular monitoring systems like Datadog, New Relic, and Prometheus. Thus, it provides a unified view of customer application performance and deep insights into their database within a comprehensive dashboard. Breaking down MongoDB’s observability offerings Tailored database performance management MongoDB employs automated tools for comprehensive database performance management, focused on real-time optimization, strategic scaling, and best practices in schema design. Using out-of-the-box tools ensures high-performing, scalable, and cost-efficient database environments ideal for modern applications. Key features include: Performance Advisor : Provides index recommendations to improve read and write performance, significantly boosting overall efficiency. Schema Advisor : Supports flexible schema design and query execution analysis to enhance performance, scalability, and validation rules for schema compliance. Opt-in Autoscaling (only available in Atlas): Optimizes resource use, manages operational costs, ensures continuous availability, and adjusts resources based on demand, preventing downtimes. Performance Advisor in action Foundational monitoring for in-depth insights MongoDB provides foundational monitoring tools and out-of-the-box insights for optimal cluster health and performance after initial database setup. These tools both help reduce the burden of performing manual tasks while laying the groundwork for detailed and granular analysis of metrics and system performance aimed at enhancing query performance, reducing execution times, and lowering resource usage. These tools include: Monitoring Charts : These charts offer detailed metrics on hardware, database operations, replication status, sharded, and search/vector search with a fine-grained metric resolution to identify issues and track trends. Real-Time Performance Panel : This panel displays live network traffic, database operations, and hardware stats, helping to identify critical operations, evaluate query times, and monitor network load and throughput. Query Insights : The recently announced Namespace Insights provides users with collection-level latency statistics. At the same time, the enhanced cluster-centric Query Profiler gives an expanded view of query performance, significantly enhancing visibility and operational efficiency across the cluster. Learn more about both! Comprehensive alerting and seamless integrations MongoDB Atlas's sophisticated alerting system offers over 200 event types, providing teams with comprehensive control and visibility over their environments. Users can fine-tune their alerting strategy with customization options to fit their specific requirements. Additionally, MongoDB Atlas enhances team collaboration and ensures a unified view of application performance through seamless integrations with third-party tools like Slack, PagerDuty, and DataDog. These integrations simplify management tasks and leverage existing workflows for optimum operational effectiveness. What’s next? Enhanced database observability Observability is more than just a technical requirement—it's a strategic asset that enhances operational efficiency and economic viability. Through MongoDB's observability suite, organizations can optimize and scale system performance and fuel innovation. MongoDB is dedicated to continuously improving this suite to manage large-scale data better and meet demanding performance standards. Our commitment is reflected in our efforts to advance MongoDB's observability features, providing specific insights that deliver actionable intelligence tailored to our customers' needs. Look for the next post in this series, where we'll explore various tools and their integration, illustrated through common use cases. Sign up for MongoDB Atlas , our cloud database service, to see database observability in action. For more information, see Monitor Your Database Deployment .

May 30, 2024

MongoDB Sales Recognized as a Top 20 Org for Professional Development by RepVue

It’s no secret that investing in your people can lead to incredible outcomes, especially for sales organizations. Companies with engaged employees see increased productivity, more profitability, and higher earnings per share . And because go-to-market (GTM) functions are responsible for more than 40% of revenue at high-growth software companies, driving effectiveness and engagement within GTM teams is paramount. At MongoDB, the sales and customer success team makes up over 30% of our company and is one of our fastest-growing organizations. We know that providing opportunities for development and receiving support from leadership drives both engagement and career growth for our teams, ultimately making them more effective and empowered to deliver results. Within the last two years, over 160 employees have been promoted into leadership roles within our sales organization, and we’re continually growing the next generation of leaders at MongoDB. So we’re thrilled to announce that MongoDB’s sales organization has received two RepVue Reppy Awards for the Spring 2024 Reppys, highlighting our commitment to excellence in building a world-class sales team and our prioritization of career development opportunities. RepVue recognizes MongoDB for being a Top 20 org for Professional Development and a Top 20 org for Publicly Funded Companies. About the RepVue Reppy awards RepVue collects millions of data points every year from its B2B sales professional user base. This data is voluntarily submitted on RepVue and includes information about their sales organization’s compensation, culture, overall employee experience, and more. This data is then used to rate the organization across categories and generate an overall RepVue Score. Sales organizations that secure a place in the Top 20 for their category or the Top 5 in a metro area become eligible to receive Reppys. These rankings serve as a testament to the exceptional sales environment these organizations consistently deliver. Hear what some of our employees have to say about working in sales at MongoDB. Adam, Enterprise Account Executive, North America “Throughout my time at MongoDB, I’ve been challenged to learn a complex technology while working with some of the smartest people in the industry. I am regularly humbled by the team we have and the engineers we work with. Everything I’ve learned from them has helped me sharpen my skills and advance my career.” Read our blog to learn more . Marie, Regional Director, Inside Sales, EMEA “Being part of the sales organization at MongoDB isn’t just about being a good sales representative, it’s about being part of a growth mindset community. Participating in our BDR to CRO career development program has helped me take my sales career to new heights.” Read our blog to learn more . Salvatore, Area Vice President, EMEA “As a member of our sales team, you’ll find a culture of transparency and meritocracy. We are focused on developing individuals to be great salespeople and like to think that we are the best technology sales school in the world. Our team is competitive, but there is a lot of camaraderie and support for one another. You’ll have the opportunity to express yourself at your best and know that you will be valued for your unique perspectives and experiences.” Read our blog to learn more . Simon, Senior Vice President, APAC “Our role as leaders is to develop our teams, from the bottom all the way up. We need to develop the next wave of leaders so that they’re prepared to step up when the time comes. Being part of a high-growth tech company requires taking risks and making mistakes. We have a high standard, and we hold each other accountable, but it never comes at the cost of creating an environment where people are afraid to fail.” Read our blog to learn more . Cedric, Chief Revenue Officer “If you join MongoDB, you’ll have to expect a culture of learning and growth. We drive a strong culture of ownership and accountability and we are going to invest in you very aggressively in terms of training and coaching, but we are also going to push you as much as we can so that you can grow your sales skills. Whether you sit in New Delhi, Sydney, Paris, London, or New York, you have a chance in this company to take on more responsibilities and grow your career faster than at any other company in the world. ” Watch the video to learn more . Stay up to date with MongoDB awards and culture highlights by joining our talent community .

May 29, 2024

A New Way to Query: Introducing the Atlas Search Playground

Today, MongoDB is thrilled to announce the launch of a brand new sandbox environment for Atlas Search. The Atlas Search Playground offers developers an unparalleled opportunity to quickly experiment, iterate, and collaborate on search indexes and queries, reducing the operational overhead throughout the entire software development lifecycle. What is the Atlas Search Playground? The Atlas Search Playground is a sandbox environment where you can explore the power and versatility of Atlas Search without needing to set up a full Atlas collection or waiting for your search index to build. It provides an instantaneous and frictionless way to experiment with creating indexes and crafting search queries on your own data—all in a single, user-friendly interface that requires no prior experience or account setup. Key Features: Instant access: No need to sign up or log in. Simply visit the Playground Environment page and start exploring immediately. Playground workspace: A dedicated workspace where you can add and modify data to work with, create, edit, and test search indexes, and test search queries in real-time. Pre-configured templates: Access a variety of sample templates to simulate real-world scenarios and test your search skills against diverse use cases. Shareable snapshots: Easily share your experiments and findings with colleagues or collaborators using unique URLs generated for each session. Just press Share to generate your unique Snapshot URL to share your pre-configured environment. A shareable snapshot from the Playground Ready to move into Atlas Search? Once you’re ready to move into Atlas, just click on the Go To Atlas button to sign up or log into your existing Atlas account. Once you are in Atlas, you can: Create a project, cluster, database, and collection to use with Atlas Search Tip! To use the documents from the Playground, select Add Documents and paste in the array of documents that you want to add. Create a search index Under the Data Services tab, click on the cluster name and navigate to the Atlas Search tab. Follow the setup instructions to create a search index. Tip! To use the search index from the Playground, select the JSON editor configuration method and paste in your index definition. Run a query Click on the name of your index, and select Search Tester from the navigation menu. Tip! To use the query from the Playground, click Edit $search query to open the query editor and paste in the query. If the query has multiple stages, click on visit the aggregation pipeline . Already an Atlas user? If you're already using Atlas Search, you can easily set up the Atlas Search Playground to match your existing configurations. All you have to do is copy and paste your documents, search index definitions, and queries into the corresponding editor panels. Ready, Set, Play Ready to embark on your search journey? Visit the Atlas Search Playground now and unleash the full potential of Atlas Search. Whether you're a seasoned pro or a curious novice, there's something for everyone to discover without the need for any setup. To learn more about the Atlas Search Playground, visit our documentation . And be sure to share what you think in our user feedback portal .

May 29, 2024

Transforming Predictive Maintenance with AI: Real-Time Audio-Based Diagnostics with Atlas Vector Search

Wind turbines are a critical component in the shift away from fossil fuels toward more sustainable, green sources of energy. According to the International Energy Agency (IEA), the global capacity of wind energy has been growing rapidly, reaching over 743 gigawatts by 2023. Wind energy, in particular, has one of the greatest potentials to increase countries' renewable capacity growth. Solar PV and wind additions are forecast to more than double by 2028 compared with 2022, continuously breaking records over the forecast period. This growth highlights the increasing reliance on wind power and, consequently, the need for effective maintenance strategies. Keeping wind turbines operating at maximum capacity is essential to ensuring their continued contribution to the energy grid. Like any mechanical device, wind turbines must undergo periodic maintenance to keep them operating at optimal levels. In recent years, advancements in technology—particularly in AI and machine learning—have played a significant role by introducing predictive maintenance breakthroughs to industrial processes like periodic maintenance. By integrating AI into renewable energy systems, organizations of all sizes can reduce costs and gain efficiencies. In this post, we will dig into an AI application use case for real-time anomaly detection through sound input, showcasing the impact of AI and MongoDB Atlas Vector Search for predictive maintenance of wind turbines. Predictive Maintenance in Modern Industries Companies increasingly invest in predictive maintenance to optimize their operations and drive efficiency. Research from Deloitte indicates that predictive maintenance can reduce equipment downtime by 5–15 percent, increase labor productivity by 5–20 percent, and reduce overall new equipment costs by 3–5 percent. This helps organizations maximize their investment in equipment and infrastructure. By implementing predictive maintenance strategies, companies can anticipate equipment failures before they occur, ultimately resulting in longer equipment lifetimes, tighter budget control, and higher overall throughput. More concretely, businesses aim to reduce mean time to repair, optimal ordering of replacement parts, efficient people management, and reduced overall maintenance costs. Leveraging data interoperability, real-time analysis, modeling and simulation, and machine learning techniques, predictive maintenance enables companies to thrive in today's competitive landscape. However, despite its immense potential, predictive maintenance also presents significant challenges. One major hurdle is the consolidation of heterogeneous data, as predictive maintenance systems often need to integrate data from various formats and sources that can be difficult to integrate. Scalability also becomes a concern when dealing with the high volumes of IoT signals generated by numerous devices and sensors. And lastly, managing and analyzing this vast amount of data in real-time poses challenges that must be overcome to realize the full benefits of predictive maintenance initiatives. At its core, predictive maintenance begins with real-time diagnostics, enabling proactive identification and mitigation of potential equipment failures in real-time. Figure 1: Predictive Maintenance starts with real-time diagnostics However, while AI has been employed for real-time diagnostics for some time, the main challenge has been acquiring and utilizing the necessary data for training AI models. Traditional methods have struggled with incorporating unstructured data into these models effectively. Enter gen AI and vector search technologies, positioned to revolutionize this landscape. Flexible data platforms working together with AI algorithms can help generate insights from diverse data types, including images, video, audio, geospatial data, and more, paving the way for more robust and efficient maintenance strategies. In this context, MongoDB Atlas Vector Search stands out as a foundational element for effective and efficient gen AI-powered predictive maintenance models. Why MongoDB and Atlas Vector Search? For several reasons, MongoDB stands out as the preferred database solution for modern applications. Figure 2: MongoDB Atlas Developer Data Platform Document data model One of the reasons why the document model is well-suited to the needs of modern applications is its ability to store diverse data types in BSON (Binary JSON) format, ranging from structured to unstructured. This flexibility essentially eliminates the middle layer necessary to convert to a SQL-like format, resulting in easier-to-maintain applications, lower development times, and faster response to changes. Time series collections MongoDB excels in handling time-series data generated by edge devices, IoT sensors, PLCs, SCADA systems, and more. With dedicated time-series collections, MongoDB provides efficient storage and retrieval of time-stamped data, enabling real-time monitoring and analysis. Real-time data processing and aggregation MongoDB's adeptness in real-time data processing is crucial for immediate diagnostics and responses, ensuring timely interventions to prevent costly repairs and downtime. Its powerful aggregation capabilities facilitate the synthesis of data from multiple sources, providing comprehensive insights into fleet-wide performance trends. Developer data platform Beyond just storing data, MongoDB Atlas is a multi-cloud developer data platform, providing the flexibility required to build a diverse range of applications. Atlas includes features like transactional processing, text-based search, vector search, in-app analytics, and more through an elegant and integrated suite of data services. It offers developers a top-tier experience through a unified query interface, all while meeting the most demanding requirements for resilience, scalability, and cybersecurity. Atlas Vector Search Among the out-of-the-box features offered by MongoDB Atlas, Atlas Vector Search stands out, enabling the search of unstructured data effortlessly. You can generate vector embeddings with machine learning models like the ones found in OpenAI or Hugging Face, and store and index them in Atlas. This feature facilitates the indexing of vector representations of objects and retrieves those that are semantically most similar to your query. Explore the capabilities of Atlas Vector Search . This functionality is especially interesting for unstructured data that was previously hard to leverage, such as text, images, and audio, allowing searches that combine audio, video, metadata, production equipment data, or sensor measurements to provide an answer to a query. Let's delve into how simple it is to leverage AI to significantly enhance the sophistication of predictive maintenance models with MongoDB Atlas. Real-time audio-based diagnostics with Atlas Vector Search In our demonstration, we'll showcase real-time audio-based diagnostics applied to a wind turbine. It's important to note that while we focus on wind turbines here, the concept can be extrapolated to any machine, vehicle, or device emitting sound. To illustrate this concept, we'll utilize a handheld fan as our makeshift wind turbine. Wind turbines emit different sounds depending on their operational status. By continuously monitoring the turbine’s audio, our system can accurately specify the current operational status of the equipment and reduce the risk of unexpected breakdowns. Early detection of potential issues allows for enhanced operational efficiency, minimizing the time and resources required for manual inspections. Additionally, timely identification can prevent costly repairs and reduce overall turbine downtime, thus enhancing cost-effectiveness. Now, let’s have a look at how this demo works! Figure 3: Application Architecture Audio Preparation We begin by capturing the audio from the equipment in different situations (normal operation, high vs. low load, equipment obstructed, not operating, etc.). Once each sound is collected, we use an embedding model to process the audio data to convert it to a vector. This step is crucial because by generating embeddings for each audio track, which are high-dimensional vector representations, we are essentially capturing the unique characteristics of the sound. We then upload these vector embeddings to MongoDB Atlas. By adding just a few examples of sounds in our database, they are ready to be searched (and essentially compared) with the sound emitted by our equipment during its operation in real-time. Audio-based diagnosis Now, we put our equipment into normal operation and start capturing the sound it is making in real-time. In this demonstration, we capture one-second clips of audio. Then, with the same embedding model used before, we take our audio clips and convert them to vector embeddings in real-time. This process happens in milliseconds, allowing us to have real-time monitoring of the audio. The one-second audio clips, now converted to vector embeddings, are then sent to MongoDB Atlas Vector Search, which can search for and find the most similar vectors from the ones we previously recorded in our audio preparation phase. The result is given back with a percentage of similarity, enabling a very accurate prediction of the current status of the operation of the wind turbine. These steps are performed repeatedly every second, leveraging fast embedding of vectors and quick searches, allowing for real-time monitoring based on sound. Check out the video below to see it in action! Transforming Predictive Maintenance with AI and MongoDB Predictive maintenance offers substantial benefits but poses challenges like data integration and scalability. MongoDB stands out as a preferred database solution, offering scalability, flexibility, and real-time data processing. As technology advances, AI integration promises to further revolutionize the industry. Thank you to Ralph Johnson and Han Heloir for their valuable contributions to this demo! Ready to revolutionize your predictive maintenance strategy with AI and MongoDB Atlas Vector Search? Try it out yourself by following the simple steps outlined in our Github repo ! Explore how MongoDB empowers manufacturing operations by visiting these resources: Generative AI in Predictive Maintenance Applications Transforming Industries with MongoDB and AI: Manufacturing and Motion MongoDB for Automotive: Driving Innovation from Factory to Finish Line

May 28, 2024