MongoDB Blog
Articles, announcements, news, updates and more
Introducing Atlas for the Edge
September 26, 2023
News
Congratulations to the 2023 EMEA MongoDB Innovation Award Winners
Throughout 2023, MongoDB has recognized the exceptional successes our APAC and North America customers realize when they build with MongoDB Atlas. Today, we are pleased to announce the recipients of the 2023 EMEA MongoDB Innovation Awards. These customers join a premiere group of organizations that have been recognized for boldly exploring the power of data and harnessing it to build the applications of tomorrow. The professionals behind these innovative projects span industries, use cases, and geographies. However, what remains constant is each organization's unfailing commitment to delivering deeper insights, powerful features, and capabilities, as well as seamless experiences, to their customers. We congratulate each of the EMEA MongoDB Innovation Award winners for delivering big results and invite you to find inspiration in their work below. Building the Next Big Thing Since being originally founded in Finland as a food delivery service, Wolt expanded their offering into groceries, gifts, and other items, and makes it possible for people living in hundreds of cities across 25 countries to get whatever they need, delivered quickly and reliably to their front door. Due to the rapidly increasing scale in their business and new partnerships with enterprise retailers, Wolt needed to rethink their existing tooling that is used by their tens of thousands of merchant partners to manage their offering inside Wolt’s delivery platform. To provide their merchants with multi-venue management capabilities, increased automation, and no limitations on amounts of menu items, Wolt chose fully-managed MongoDB Atlas due to its flexible data document model, support for Kafka and event-driven architectures, high performance, and limitless scalability. Industry Transformation Gong ’s Revenue Intelligence Platform uses proprietary AI technology to enable revenue teams to utilize customer interactions to increase business efficiency, improve decision-making, and accelerate revenue growth. Gong chose MongoDB Atlas as a high-performing transactional database due to its ability to handle Gong’s complex data structures and algorithms, and indeed MongoDB Atlas was able to support 30-40 million daily interactions and mission-critical search queries. Through the adoption of MongoDB Atlas, Gong’s queries take only 20-30 milliseconds, and with Atlas’ sharding capabilities, Gong future-proofed its environment for ongoing growth and gained greater control over its customer data. Innovative Leadership in Diversity, Equity, and Inclusion As part of its "Inclusion for All" strategy, Vodafone is committed to ensuring our digital society is accessible to all, and that women feel empowered to participate in it. In addition, Vodafone’s mission is to have 40% of women in management roles by 2030. Earlier this year, Vodafone UK teamed up with MongoDB to host interactive sessions and networking for “Digital & IT Women in Tech Week.” With a shared goal of reducing the gender gap in technology and inspiring young women to build their careers by pursuing leadership roles, the week’s events brought together more than 200 Vodafone employees to discuss breaking workplace biases and enhancing gender equality in the industry. In addition, Vodafone spoke at the 2023 MongoDB.local London event in the IDEA Lounge. Inspiring Innovation Ernst & Young LLP (EY) manages high volumes of transactional data and its clients and internal teams work under strict timelines to file taxes and meet regulatory deadlines. Their cloud-based Global VAT Reporting Tool (GVRT) platform automates and digitizes the preparation of 242 different types of returns across 79 countries. With support from MongoDB Professional Services, EY planned and executed a data migration from their previous database solution to MongoDB Atlas so that the GVRT tool could fully benefit from MongoDB’s developer data platform. Since migrating to Atlas, EY experienced a significant performance boost, they reduced costs by as much as 50%, and they are able to scale without limitations to handle increased data volumes, transactional loads, and concurrent user requests during peak periods. Find out Who Uses MongoDB and read more about our customers’ MongoDB Atlas successes.
Recap of Product Announcements at MongoDB.local London, 2023
This post is also available in: Deutsch , Français , 中文 , Español , Português . We’re now more than three months into our MongoDB.local world tour that kicked off in NYC earlier this June. Since then, we’ve continued to introduce product enhancements and new capabilities, from the GA of MongoDB for VS Code to MongoDB 7.0 and Queryable Encryption . Today, I’m excited to share the highlights of recent product announcements from our London conference this morning. Efficient and intelligent developer experiences for building with MongoDB We’ve always been committed to providing the best developer experience because we know that developer time is one of the most precious commodities in any organization. When we looked at the most common tasks developers perform on a daily basis, we recognized two areas for improvement: making development against Atlas more efficient and making it easier to write MongoDB queries. We want to give developers the most ergonomic way to work with MongoDB Atlas throughout their entire journey. For many developers, that journey begins by working with MongoDB locally before moving to the cloud - which is why we’re investing in a great local development experience. Starting today , developers can use the Atlas CLI to manage local development environments with the same experience as Atlas clusters in the cloud. Beyond making it easy to deploy and manage development instances, we also want to bring the breadth of our developer data platform to local environments. The new Atlas CLI experience, available in public preview, also comes with integrated Atlas Search and Atlas Vector Search so developers can create and manage search indexes and queries within their development workflows. This is the first of more investments to come as we continue to build a seamless experience for services in Atlas from sandbox to testing and production. The other problem we want to solve is speed, and we’re excited to use generative AI technology to introduce several new intelligent developer experiences . Querying data should be as easy as asking a question in a language that feels natural to you. Developers can now ask questions in plain English and Compass , our MongoDB GUI, will generate the corresponding query in MongoDB query language syntax. From simple queries to more complex aggregations, this experience will reduce the friction of learning MongoDB’s query language and help developers iterate and build new features more quickly. We’re also introducing a new language interface for Atlas Charts so developers can easily visualize data in MongoDB and an AI chatbot for our documentation resources. For customers embarking on a migration journey from using relational databases to using MongoDB, one of the most difficult and important steps is converting hundreds, if not thousands, of queries and application code. Available now in private preview, SQL query conversion in Relational Migrator can convert queries and stored procedures to MongoDB query language syntax at scale, shifting resources from query creation to review and implementation. Run MongoDB anywhere - from edge to cloud One of the benefits of MongoDB that we’ve been proud of since the beginning is the flexibility to build with it anywhere - on a local machine for development, fully managed across multiple public clouds , on-premises or in a private cloud, and even on mobile and edge devices. As mobility and IoT become more essential to operations across industries, one of the key requirements is being able to sync and move data across environments. Today , we’re excited to announce Atlas for the Edge , which brings data processing and storage capabilities closer to where it’s often most needed - right where data is generated. With Atlas Edge Servers that can be deployed anywhere and built-in conflict resolution, customers can easily create hub and spoke architectures to power customer experiences that require ultra-low latency or heavier computation close to where data is generated. From manufacturing to retail to healthcare , Atlas for the Edge enables customers to unlock more use cases that rely on a connected data layer across public clouds, on-premise or edge computing locations, and sensors and devices. Build the next generation of AI-powered applications with a developer data platform Since our public preview announcement earlier this year, we’ve seen a lot of interest in Atlas Vector Search, particularly in building RAG (retrieval augmented generation) architectures for applications powered by Generative AI . From startups to established companies, customers are eager to build more intelligent applications with the backing of a modern, highly scalable, and performant platform. The ability to store vector embeddings alongside source and metadata has simplified how developers build GenAI into new and existing applications, and with the introduction of the $vectorSearch aggregation stage, it will be even easier to pre-filter and tune results using the MongoDB query language, all in a single platform on Atlas. Finally, we recognize the need to empower developers with practical resources to expand their skills and knowledge. In addition to new content available on MongoDB University , we announced MongoDB Press , a medium for publishing technical and leadership knowledge about MongoDB. The first two books are on aggregations and mastering MongoDB 7.0. We also added a solutions library on our website with use cases organized by industry verticals to show the art of what’s possible with our developer data platform. To see more announcements and get the latest product updates, visit our What’s New page. Head to the MongoDB.local hub to see where we'll be showing up next.
New Intelligent Developer Experiences for Compass, Atlas Charts, Relational Migrator, and Docs
This post is also available in: Deutsch , Français , 中文 , Español , Português . Today, MongoDB announced a range of innovations in its developer data platform, creating new, intelligent developer experiences in familiar tools like MongoDB Compass, Atlas Charts, Relational Migrator, and MongoDB Documentation that radically simplify and accelerate how developers build modern applications. These new experiences provide developers with guided and intelligent assistance for their development processes in: MongoDB Compass: Where developers can use natural language to compose everything from simple queries to sophisticated, multi-stage aggregations. MongoDB Relational Migrator: Where developers can convert SQL queries to MongoDB Query API syntax. MongoDB Atlas Charts: Where developers can use natural language to generate basic data visualizations. MongoDB Documentation: Where developers can ask questions to an intelligent chatbot, built on top of MongoDB Atlas and Atlas Vector Search, to enable lightning-fast information discovery and troubleshooting during software development. Developer time is one of the most precious commodities in any organization, and with business and customer expectations continuing to rise, developers are under increasing pressure to deliver applications quickly. With more intelligent experiences across the MongoDB developer data platform, it is now simpler and easier than ever to build modern applications for virtually any use case. Natural Language Queries in Compass Building queries and aggregations is one of the most prominent developer use cases for Compass , MongoDB’s popular, downloadable GUI tool. Compass’ new, intelligent experience allows developers to use natural language to compose sophisticated aggregations to query, transform, and enrich data, reducing the complexity and learning curve to build queries into application code. The new experience is being released in Public Preview in version 1.40.0 and will be rolled out incrementally to users starting today until the end of October. To get started, make sure you have 1.40.0 downloaded on your machine and have access to the feature. Then you can navigate to the Documents tab and click on the Generate Query button in the query bar, which opens a second bar below the standard query bar where you can enter natural language prompts to generate the Query API syntax for you to execute against your data. Be sure to hit the “thumb’s up” or “thumb’s down” button to rate the helpfulness of the query generated. SQL Query Conversion in Relational Migrator Migrations are part of many developers’ journeys with MongoDB. Earlier this summer at MongoDB.Local NYC, we announced Relational Migrator to help teams with these projects, and we’re continuing to make it easier to modernize application code. Many legacy systems have hundreds, if not thousands of SQL queries that must be modernized as part of any migration effort, and that can be a time-consuming, if not daunting task. Now in Private Preview, developers can use Relational Migrator to convert existing SQL queries and stored procedures into development-ready MongoDB Query API syntax. With SQL query conversion, developers can leverage Relational Migrator to eliminate the manual effort of creating MongoDB queries at scale - speeding up migration projects. SQL query conversion is currently available in Private Preview, and access can be requested directly from the latest version of Relational Migrator. Natural Language Support in Atlas Charts Atlas Charts is the best way for developers to visualize Atlas data. By offering an effortless and powerful solution for gaining data-driven insights, Charts empowers developers and the businesses they help scale. What has always been easy is now becoming more intelligent too! Available in Private Preview, a new natural language mode allows developers to visualize their data through a simple language query, for example: “show me a comparison of annual revenue by country and product.” This is just the start. Later this year, natural language support will extend to more complex queries and chart types. Sign up today to try out natural language support for building charts! Stay tuned for more updates from the team and check out our documentation to learn more about what’s supported by natural language during Private Preview! Intelligent Chatbot for MongoDB Documentation Documentation is critical to the developer experience, making it easier to discover product features and capabilities and troubleshoot common challenges during software development. MongoDB is now super-charging your experience with an intelligent chatbot that improves information discovery by surfacing and summarizing the most relevant documentation. Built with MongoDB Atlas and Atlas Vector Search, the chatbot allows you to ask questions in natural language like “How do I get started with MongoDB Atlas?” or “How do I add a new IP address to the IP access list for my Atlas project?” and receive a response with reference articles, code examples, and other relevant information. MongoDB will also be open-sourcing and providing educational materials about how we built the intelligent chatbot, making it that much easier for others in the community to use the power of MongoDB Atlas and Atlas Vector Search to create dynamic and educational experiences for their end users. Data Privacy and Security MongoDB is trusted by some of the world's most security-conscious organizations, who use the developer data platform’s robust data security and privacy controls to manage their most sensitive data assets. To maintain this trust, these new developer experiences will always be transparent about what data is accessed and used, allowing customers to make informed decisions within the boundaries of their unique security, privacy, and compliance concerns. Get Started Today With new, intelligent features that allow developers to interact with their data using natural language in Compass, Relational Migrator, and Charts, as well as an intelligent chatbot for MongoDB Documentation, it’s easier than ever to take advantage of the flexibility and scalability of MongoDB's document data model to build any class of application. If you have feedback on these experiences, you can enter a suggestion in our user feedback portal .
Introducing a Local Experience for Atlas, Atlas Search, and Atlas Vector Search with the Atlas CLI
This post is also available in: Deutsch , Français , 中文 , Español , Português . Today, MongoDB is pleased to announce in Public Preview a new set of features for building software locally with MongoDB Atlas, giving developers greater flexibility and reducing operational overhead throughout the entire software development lifecycle. Developers can now develop locally with MongoDB Atlas deployments, including Atlas Search and Vector Search , using the Atlas CLI , empowering them to create full-text search or AI-powered applications no matter their preferred environment for building with MongoDB. Developers can use the Atlas CLI to set up, connect to, and automate common management tasks from early development through testing, staging, and production. For full-text search use cases, developers can now use the Atlas CLI to create and manage Atlas Search indexes regardless of whether they are working locally or in the cloud. Similarly, developers building applications powered by semantic search and generative AI on MongoDB can now use the Atlas CLI to create and manage local development instances with Vector Search indexes regardless of their development environment. Developer time is one of the most precious commodities in any organization building innovative new application experiences. But all too frequently, developers are burdened with managing repeatable tasks such as setting up development environments. They also often have to wrestle with the cognitive overhead of switching between different user experiences for local versus cloud development, distracting from delivering value. By giving developers the power of Atlas at their fingertips no matter their preferred development environment, MongoDB continues to expand the scope and capabilities of its developer data platform while placing a premium on developer experience. Create a Local Atlas Database Ready to create a local Atlas database, but don’t have the Atlas CLI yet? It’s easy to install with your favorite package manager. To install the Atlas CLI with Homebrew, use the following command: brew install mongodb-atlas In addition to installing via the Homebrew package manager, you can install the MongoDB Atlas CLI via Apt, Yum, Chocolatey, directly downloading the binary, or pulling the Docker image (learn more about our documentation ). You can also download it directly from the MongoDB Download Center . To create a local Atlas deployment with default settings in interactive mode, enter: atlas deployments setup --type local If you want to list your Atlas deployments enter: atlas deployments list If you’re authenticated to Atlas, you will see both your local and cloud Atlas deployments. If you aren’t authenticated to Atlas, you will only see your local deployments. Get Started with Local Atlas Search Building an application with a full-text search feature powered by Atlas Search? If you’re a developer who tends to build and prototype locally, you may be interested in using the Atlas CLI to work with Atlas Search in your local environment. To get started, first, connect to the local deployment on which you’d like to create a Search index: atlas deployments connect Next, you can use the MongoDB Shell to create your Search index. Below you’ll see an example of how to create an Atlas Search index: db.YOURCOLLECTION.createSearchIndex( "example-index", { mappings: { dynamic: true } } ) Then, if you want to run a query you can use the $search stage of an aggregation pipeline. You can learn more about managing Atlas Search indexes in our documentation . Get Started with Local Vector Search If you’re building an application with generative AI or semantic search and MongoDB Atlas, chances are you’ll be interested in our Atlas Vector Search offering. And now with the Atlas CLI, you can work with Vector Search in the cloud and your local environment. To get started with Vector Search locally you can use MongoDB Shell to create a Vector Search index. Notice that this is similar to the Atlas Search example above, except that in this case there is a vector embedding accounted for in search index creation. db.YOURCOLLECTION.createSearchIndex({ "mappings": { "dynamic": true, "fields": { "plot_embedding": { "type": "knnVector", "dimensions": 1536, "similarity": "euclidean" } } } } ) To learn more about running Vector Search queries visit our documentation . Additionally, if you're already familiar with handling your cloud Search indexes using the Atlas CLI, you'll appreciate a fresh set of interactive commands designed to help you efficiently manage Atlas Search and Vector Search indexes both locally and in the cloud: atlas deployments search indexes create From there you can move through an interactive flow that guides you through index creation. For detailed instructions visit our tutorial . Ready to Move to the Cloud? If you’re ready to create an Atlas database in the cloud, that is easy to do with the Atlas CLI. Simply use the following command: atlas deployments setup --type atlas From there, the setup wizard will guide you to: Register for an Atlas account or authenticate to an existing account Create a free MongoDB Atlas database Load sample data Add your IP address to the access list Create a database user and password Connect to the cluster using the MongoDB Shell ( mongosh ) so you can begin interacting with your data To learn more about the Atlas CLI, visit our documentation . And be sure to let us know what you think of the Atlas CLI in our user feedback portal . With the new local experience with the Atlas CLI, it’s easier than ever to work with your data on Atlas no matter your preferred development environment. Get started today with the Atlas CLI as the ultimate developer tool to manage MongoDB Atlas, including Atlas Search and Vector Search, throughout the entire software development lifecycle, from your local environment all the way to the cloud.
Realm is Now Atlas Device SDKs
This post is also available in: Deutsch , Français , 中文 , Español , Português . Change is essential to growth and progress in open source technology. MongoDB is announcing today that we’re renaming Realm to MongoDB Atlas Device SDKs. We will continue offering MongoDB Atlas Device SDKs as a free and open source project under Apache License 2.0. MongoDB acquired Realm and its technology in 2019 and has continued development of the project to provide developers a synchronized data layer between devices and the cloud that makes it easier to build mobile applications, including support for multiple programming languages, development frameworks, and cloud providers. MongoDB will continue open source development of Realm as Atlas Device SDKs, and developers are free to use the project — with or without MongoDB Atlas — to build reactive mobile applications using the technology of their choice. Regular updates to the project will continue to be available on its Github repository . Please engage with us on the MongoDB Community Forum with questions or feedback.
Hot Off the Press: MongoDB Launches Two New Books at MongoDB.local London
It’s well known that developers today are facing immense demand to build new, modern applications at an accelerated pace. In fact, recent data from IDC predicts that by 2025 there will be a shortfall of 4 million developers . As the industry moves towards more complex and advanced technologies, developers remain the foundation – as well as the key – to expanding emerging, innovative technologies, from AI to IoT and other automation applications. The skills required to develop these technologies are becoming increasingly specialized. In turn, the need for accessible resources, training, and education is only becoming more pressing. That’s why we’re delighted to launch MongoDB Press – our very own official series of educational books – penned by a mix of our in-house experts and trusted industry voices, covering both technical and strategic topics. We’re thrilled that our first two books of the series – Mastering MongoDB 7.0 and Practical MongoDB Aggregations – will be launched at MongoDB .local London 2023 . Practical MongoDB Aggregations was written by our very own Executive Solutions Architect, Paul Done, and is intended for developers with a baseline understanding of the MongoDB Aggregation Framework. The book will allow readers to learn more about building aggregation pipelines. You can get 20% off now by visiting mongodb.com/books . Attendees at MongoDB .local London will be able to get signed copies of the book while supplies last Mastering MongoDB 7.0 is available for pre-order and was written by a team of MongoDB experts. It provides a deep-dive into the latest features of MongoDB. By the end of the book, readers will have gained the practical understanding required to design, develop, administer, and scale MongoDB-based database applications, both on-premises and on the cloud.
Being Latine in Tech: Two MongoDB Employees Share Their Advice on Building Careers in Engineering
Ashley Naranjo and Martin Bajana, members of MongoDB’s employee resource group QueLatine, share their career journeys and offer insight into how other members of the Latine community can build careers in tech. Jackie Denner: How did you make your way into the tech industry? Ashley Naranjo: I am a first-generation Latina with a passion for Information Technology and a knack for problem-solving. After graduating early from high school, I embarked on a career in Nursing. I chose Nursing initially because I wanted to make a difference and help others, but my path took an unexpected turn when COVID-19 reshaped our world. In light of the circumstances, I reevaluated my options and decided to seize an opportunity with a program called Year Up . During the intensive six-month training and deployment phase, I not only completed rigorous coursework but also obtained IT Google Coursera certifications and actively pursued CompTIA certifications. This experience allowed me to secure an internship at Meta (Facebook) as an Enterprise Operation IT Support Tech, where my love for technology blossomed. During my time at Meta, I had the privilege of assisting diverse Meta users worldwide with a wide range of technical issues, including troubleshooting, software and hardware support, internal access permissions, and more. The exposure to a global tech environment further fueled my passion for the field. When my internship concluded, I was offered a 1-year contract role with Meta to continue my work as a support tech for the same team. Throughout that year, I immersed myself in all aspects of technology, maximizing my learning opportunities and applying my networking skills. As time went on, I knew I needed a new challenge. This led me to embark on a search for an exciting role, which eventually brought me to MongoDB. I am passionate about driving technological innovation, and MongoDB is a place where I can make an impact. Martin Bajana: My interest in technology stems from a variety of sources. From a young age, I developed a strong passion for video games and exploring new technologies. Whether it was experimenting with the latest gaming consoles or delving into computer hardware, I relished the opportunity to learn and understand the inner workings of these technologies. In school, I discovered my affinity for mathematics, which further solidified my decision to pursue a career in the tech industry. Choosing to study computer science in college was a natural progression for me, as it allowed me to combine my love for technology with my aptitude for problem-solving. After completing my education, I was recruited by Verizon, where I worked on front-end applications and Android development. Although the transition was initially challenging, I persevered and regained my confidence. It was during this period that I realized a career in technology was my long-term aspiration. Throughout my tenure at Verizon, I embraced opportunities to work across various teams, acquiring valuable experience and honing my skills. Eventually, I made the decision to join MongoDB, which has provided me with an enriching journey and the chance to shape my career in the tech industry. JD: Have there been any challenges you've faced throughout your career? AN: Imposter syndrome has been a significant challenge for me throughout my career, and it's something I still deal with to this day. When surrounded by my talented colleagues, I would often compare myself to them and focus on my perceived weaknesses and flaws, leading to a lack of self-confidence. However, I tackled this issue by addressing my feelings with my manager. Her support and guidance helped me realize my own potential and acknowledge my accomplishments. Maintaining a positive mindset has enabled me to view myself as a competent engineer and recognize the value I bring to my team. I have learned to take ownership of my successes and embrace opportunities for growth. Stepping out of my comfort zone has become a regular practice, as personal and professional development often stems from embracing challenges and discomfort. By giving myself permission to take up space and be confident in my abilities, I have been able to overcome imposter syndrome and continue to thrive in my role. MB: I have been fortunate enough to work for companies and teams that value and respect me for the work I deliver. Being in the tech industry and growing up in a culturally diverse region of the country, I have had exposure to individuals from various backgrounds and identities, which has made me more comfortable as a Latinx individual in the industry. My personal goal is to promote a work environment where everyone is judged based on the contributions they bring to the team, rather than their identity. I believe in supporting and respecting the identities of my peers and coworkers while fostering a culture of inclusivity and equality. JD: How has MongoDB supported your career growth and development? AN: In my time working at MongoDB, I have experienced exceptional support that has greatly contributed to my professional development and growth. As an engineer at MongoDB, I have been provided with numerous opportunities to expand my knowledge and skills through participation in tech talks, hackathons, and continuous learning about emerging technologies. I am grateful for the proactive approach taken by my manager and team leaders in fostering my growth as an engineer. Additionally, MongoDB's commitment to diversity and inclusion is evident through the company's DEI initiatives. Platforms like our employee resource group “QueLatine” have made me feel a stronger sense of connection and belonging, particularly among my Latinx peers. By recognizing the power of our diverse backgrounds and experiences, MongoDB empowers us to have a meaningful impact in the industry. MB: I have experienced full support from my leader since day one. They have proactively sought to understand my career goals and have helped me create a clear career path to achieve those goals. This level of support has enabled me to take on challenging projects and initiatives within the company, allowing me to grow and develop in my career. Furthermore, MongoDB offers a wealth of learning and development resources to its employees, which I have fully utilized to continue learning and growing my skill set. JD: What is your advice for other Latines who want to begin careers in tech? AN: Having made a significant career change myself, I can empathize with the challenges that come with exploring new paths, particularly in the tech industry. As a Latina in tech, I feel a strong desire to encourage and raise awareness within our community about the incredible resources and opportunities that are available to us. My advice to others who may be considering a similar journey is to prioritize the continuous development of your technical skills, actively seek out mentoring opportunities, push yourself beyond your comfort zone by honing your networking abilities, and most importantly, believe in yourself and your ability to achieve great things! MB: Navigating the vast world of technology can certainly be overwhelming, but it's important not to fear feeling lost. Even after 12 years in this career, there are still days where I come across something I've never heard of before. Fortunately, we live in a world abundant with resources for continuous learning. My advice is to take the time to explore and ask questions. Seek out open-source projects that you can contribute to, and connect with other professionals in the tech industry who can share their experiences and provide guidance. Additionally, taking advantage of hackathons and other tech events can expose you to new technologies and ideas. Don't be afraid to make mistakes, and most importantly, don't give up! Join us in transforming the way developers work with data. Build your tech career at MongoDB .
Fusing MongoDB and Databricks to Deliver AI-Augmented Search
With customers' attention more and more dispersed across channels, platforms, and devices, the retail industry rages with the relentless competition. The customer’s search experience on your storefront is the cornerstone of capitalizing on your Zero Moment of Truth, the point in the buying cycle where the consumer's impression of a brand or product is formed. Imagine a customer, Sarah, eager to buy a new pair of hiking boots. Instead of wandering aimlessly through pages and pages of search results, she expects to find her ideal pair easily. The smoother her search, the more likely she is to buy. Yet, achieving this seamless experience isn't a walk in the park for retailers. Enter the dynamic duo of MongoDB and Databricks. By equipping their teams with this powerful tech stack, retailers can harness the might of real-time in-app analytics. This not only streamlines the search process but also infuses AI and advanced search functionalities into e-commerce applications. The result? An app that not only meets Sarah's current expectations but anticipates her future needs. In this blog, we’ll help you navigate through what are the main reasons to implement an AI-augmented search solution by integrating both platforms. Let’s embark on this! A solid foundation for your data model For an e-commerce site built around the principles of an Event Driven and MACH Architecture , the data layer will need to ingest and transform data from a number of different sources. Heterogeneous data, such as product catalog, user behavior on the e-commerce front-end, comments and ratings, search keywords, and customer lifecycle segmentation- all of this is necessary to personalize search results in real time. This increases the need for a flexible model such as in MongoDB’s documents and a platform that can easily take in data from a number of different sources- from API, CSV, and Kafka topics through the MongoDB Kafka Connector . MongoDB's Translytical capabilities, combining transactional (OLTP) and analytical (OLAP) offer real-time data processing and analysis, enabling you to simplify your workloads while ensuring timely responsiveness and cost-effectiveness. Now the data platform is servicing the operational needs of the application- what about adding in AI? Combining MongoDB with Databricks, using the MongoDB Spark Connector can allow you to train your models with your operational data from MongoDB easily and to trigger them to run in real-time to augment your application as the customer is using it. Centralization of heterogeneous data in a robust yet flexible Operational Data Layer The foundation of an effective e-commerce data layer lies in having a solid yet flexible operational data platform, so the orchestrating of ML models to run at specific timeframes or responding to different events, enabling crucial data transformation, metadata enrichment, and data featurization becomes a simple, automated task for optimizing search result pages and deliver a frictionless purchasing process. Check out this blog for a tutorial on achieving near real-time ingestion using the Kafka Connector with MongoDB Atlas, and data processing with Databricks Spark User Defined Functions. Adding relevance to your search engine results pages To achieve optimal product positioning on the Search Engine Results Page (SERP) after a user performs a query, retailers are challenged with creating a business score for their products' relevance. This score incorporates various factors such as stock levels, competitor prices, and price elasticity of demand. These business scores are complex real-time analyses calibrated against so many factors- it’s a perfect use case for AI. Adding AI-generated relevance to your SERPs can accurately predict and display search results that are most relevant to users' queries, leading to higher engagement and increased click-through rates, while also helping businesses optimize their content based on the operational context of their markets. The ingestion into the MongoDB Atlas document-based model laid the groundwork for this challenge, and leveraging the MongoDB Apache Spark Streaming Connector companies can persist their data into Databricks, taking advantage of its capabilities for data cleansing and complex data transformations, making it the ideal framework for delivering batch training and inference models. Diagram of the full architecture integrating MongoDB Atlas and Databricks for an e-commerce store, real-time analytics, and search MongoDB App Services act as the mortar of our solution, achieving an overlap of the intelligence layer in an event-driven way, making it not only real-time but also cost-effective and rendering both your applications and business processes nimble. Make sure to check out this GitHub repository to understand in depth how this is achieved. Data freshness Once that business score can be calculated comes the challenge of delivering it over the search feature of your application. With MongoDB Atlas native workload isolation, operational data is continuously available on dedicated analytics nodes deployed in the same distributed cluster, and exposed to analysts within milliseconds of being stored in the database. But data freshness is not only important for your analytics use cases, combining both your operational data with your analytics layer, retailers power in-app analytics and build amazing user experiences across your customer touch points. Considering MongoDB Atlas Search 's advanced features such as faceted search, auto-complete, and spell correction, retailers rest assured of a more intuitive and user-friendly search experience not only for their customers but for their developers, as it minimizes the tax of operational complexity as all these functionalities are bundled in the same platform. App-driven analytics is a competitive advantage against traditional warehouse analytics Additionally, the search functionality is optimized for performance, enabling businesses to handle high search query volumes without compromising user experience. The business score generated from the AI models trained and deployed with Databricks will provide the central point to act as a discriminator over where in the SERPs any of the specific products appear, rendering your search engine relevance fueled and securing the delivery of a high-quality user experience. Conclusion Search is a key part of the buying process for any customer. Showing customers exactly what they are looking for without investing too much time in the browsing stage reduces friction in the buying process, but as we’ve seen it might not be so easy technically. Empower your teams with the right tech stack to take advantage of the power of real-time in-app analytics with MongoDB and Databricks. It’s the simplest way to build AI and search capabilities into your e-commerce app, to respond to current and future market expectations. Check out the video below and this GitHub repository for all the code needed to integrate MongoDB and Databricks and deliver a real-time machine-learning solution for AI-augmented Search.
Why Queryable Encryption Matters to Developers and IT Decision Makers
Enterprises face new challenges in protecting data as modern applications constantly change requirements. There are new technologies, advances in cryptography, regulatory constraints, and architectural complexities. The threat landscape and attack techniques are also changing, making it harder for developers to be experts in data protection. Client-side field level encryption , sometimes referred to as end-to-end encryption, provides another layer of security that enables enterprises to protect sensitive data. Although client-side encryption fulfills many modern requirements, architects, and developers face challenges in implementing these solutions to protect their data efficiently for several reasons: Multiple cryptographic tools to choose from — Identifying the relevant libraries, selecting the appropriate encryption algorithms, configuring the selected algorithms, and correctly setting up the API for interaction are some of the challenges around tools. Encryption key management challenges — how and where to store the encryption keys, how to manage access, and how to manage key lifecycle such as rotation and revocation. Customize application(s) — Developers might have to write custom code to encrypt, decrypt, and query the data requiring widespread application changes. With Queryable Encryption now generally available, MongoDB helps customers protect data throughout its data lifecycle — data is encrypted at the client side and remains encrypted in transit, at rest, and in use while in memory, in logs, and backups. Also, MongoDB is the only database provider that allows customers to run rich queries on encrypted data, just like they can on unencrypted data. This is a huge advantage for customers as they can query and secure the data confidently. Why does Queryable Encryption matter to IT decision-makers and developers? Here are a few reasons: Security teams within enterprises deal with protecting their customers’ sensitive data — financial records, personal data, medical records, and transaction data. Queryable Encryption provides a high level of security — by encrypting sensitive fields from the client side, the data remains encrypted while in transit, at rest, and in use and is only ever decrypted back at the client. With Queryable Encryption, customers can run expressive queries on encrypted data using an industry-first fast, encrypted search algorithm. This allows the server to process and retrieve matching documents without the server understanding the data or why the document should be returned. Queryable Encryption was designed by the pioneers of encrypted search with decades of research and experience in cryptography and uses NIST-standard cryptographic primitives such as AES-256, SHA2, and HMACs. Queryable Encryption allows a faster and easier development cycle — developers can easily encrypt sensitive data without making changes to their application code by using language-specific drivers provided by MongoDB. There is no crypto experience required and it’s intuitive and easy for developers to set up and use. Developers need not be cryptography experts to encrypt, format, and transmit the data. They don't have to figure out how to use the right algorithms or encryption options to implement a secure encryption solution. MongoDB has built a comprehensive encryption solution including key management. Queryable Encryption helps enterprises meet strict data privacy requirements such as HIPAA, GDPR, CCPA, PCI, and more using strong data protection techniques. It offers customer-managed and controlled keys. The MongoDB driver handles all cryptographic operations and communication with the customer-provisioned key provider . Queryable Encryption supports AWS KMS, Google Cloud KMS, Azure Key Vault, and KMIP-compliant key providers. MongoDB also provides APIs for key rotation and key migration that customers can leverage to make key management seamless. ** Equality query type is supported in 7.0 GA *With automation encryption enabled For more information on Queryable Encryption, refer to the following resources: Queryable Encryption documentation Queryable Encryption FAQ Download drivers Queryable Encryption Datasheet
View and Analyze Your Monthly MongoDB Atlas Usage with Cost Explorer
In today's macroeconomic climate, knowing where your money's going is a big deal. From optimizing costs to boosting efficiency, understanding your software expenses can be a total game-changer for your business. That’s why we’re excited to announce the release of Cost Explorer in MongoDB Atlas. Cost Explorer is a new visual interface available in the Billing section of the Atlas UI that is meant to help you view and analyze your monthly MongoDB Atlas usage in one convenient location. How can Cost Explorer help you? Cost Explorer allows you to easily filter your Atlas usage data by what’s most important to you and your business, with filters to segment your view by organization (if you have cross-org billing enabled), projects, clusters, or services, within a time window of up to 18 months. With Cost Explorer, you can now quickly pinpoint trends or outliers in your month-over-month usage to identify opportunities to potentially improve or optimize your Atlas usage going forward. If you’re looking for additional customization beyond what is available in Cost Explorer, you can also create your own billing dashboards in Atlas Charts that are fully tailored to your needs. Cost Explorer is viewable for any Atlas user assigned the Organization Owner, Billing Admin, or Organization Billing Viewer roles. To learn more about Cost Explore and how to manage your Atlas billing, view our documentation on managing billing .
How MongoDB and Alibaba Cloud are Powering the Era of Autonomous Driving
The emergence of autonomous driving technologies is transforming how automotive manufacturers operate, with data taking center stage in this transformation. Manufacturers are now not only creators of physical products but also stewards of vast amounts of product and customer data. As vehicles transform into connected vehicles, automotive manufacturers are compelled to transform their business models into software-first organizations. The data generated by connected vehicles is used to create better driver assistance systems and paves the way for autonomous driving applications. It has to be noted that the journey toward autonomous vehicles is not just about building reliable vehicles but harnessing the power of connected vehicle data to create a new era of mobility that seamlessly integrates cutting-edge software with vehicle hardware. The ultimate goal of autonomous vehicle makers is to produce cars that are safer than human-driven vehicles. Since 2010, investors have poured over 200 billion dollars into autonomous vehicle technology. Even with this large amount of investment, it is very challenging to create fully autonomous vehicles that can drive safer than humans. Some experts estimate that the technology to achieve level 5 autonomy is about 80% developed but the last 20% will be extremely hard to achieve and will take a lot of time to perfect. Unusual events such as extreme weather, wildlife crossings, and highway construction are still enigmas for many automotive companies to solve. The answer to these challenges is not straightforward. AI-based image and object recognition still has a long way to go to deal with uncertainties on the road. However, one thing is certain, automotive manufacturers need to make use of data captured by radar, LiDAR, camera systems, and the whole telemetry system in the vehicle in order to train their AI models better. A modern vehicle is a data powerhouse. It constantly gathers and processes information from onboard sensors and cameras. The Big Data generated as a result presents a formidable challenge, requiring robust storage and analysis capabilities. Additionally, this time series data needs to be analyzed in real-time and decisions have to be made instantaneously in order to guarantee safe navigation. Furthermore, ensuring data privacy and security is also a hurdle to cross since self-driving vehicles need to be shielded from cyber attacks as such an attack can cause life-threatening events. The development of high-definition (HD) maps to help the vehicle ‘see’ what is on the road also poses technical challenges. Such maps are developed using a combination of different data sources such as Global Navigation Satellite Systems (GNSS), radar, IMUs, cameras, and LiDAR. Any error in any one of these systems aggregates and ultimately impacts the accuracy of the navigation. It is required to have a data platform in the middle of the data source (vehicle systems) and the AI platform to accommodate and consolidate this diverse information while keeping this data secure. The data platform should be able to preprocess this data as well as add additional context to it before using it to train or run the AI modules such as object detection, semantic segmentation, and path planning. MongoDB can play a significant role in addressing above mentioned data-related challenges posed by autonomous driving. The document model is an excellent way to accommodate diverse data types such as sensor readings, telematics, maps, and model results. New fields to the documents can be added at run time, enabling the developers to easily add context to the raw telemetry data. MongoDB’s ability to handle large volumes of unstructured data makes it suitable for the constant influx of vehicle-generated information. MongoDB is not only an excellent choice for data storage but also provides comprehensive data pre-processing capabilities through its aggregation framework. Its support for time series window functions allows data scientists to produce calculations over a sorted set of documents. Time series collections also dramatically reduce storage costs. Column compression significantly improves practical compression, reduces the data's overall storage on disk, and improves read performance. MongoDB offers robust security features such as role-based access control, encryption at rest and in transit, comprehensive auditing, field-level redaction and encryption, and down to the level of client-side field-level encryption that can help shield sensitive data from potential cyber threats while ensuring compliance with data protection regulations. For challenges related to effectively storing and querying HD maps, MongoDB’s geospatial features can aid in querying location-based data and also combining the information from maps with telemetry data fulfilling the continuous updates and accuracy requirements for mapping. Furthermore, MongoDB's horizontal scaling or sharding allows for the seamless expansion of storage and processing capabilities as the volume of data grows. This scalability is essential for handling the data streams generated by fleets of self-driving vehicles. During the research and development of autonomous driving projects, scalable infrastructure is required to quickly and steadily collect and process massive data. In such projects, data is generated at the terabyte level every day. To meet these needs, Alibaba Cloud provides a solution that integrates data collection, transmission, storage, and computing. In this solution, the data collected daily by sensors can be simulated and collected using Alibaba Cloud Lightning Cube and sent to the Object Storage Service (OSS) . Context is added to this data using a translator and then this contextualized information can be pushed to MongoDB to train models. MongoDB and Alibaba Cloud recently announced a four-year extension to their strategic global partnership that has seen significant growth since being announced in 2019. Through this partnership, automotive manufacturers can easily set up and use MongoDB-as-a-service-AsparaDB for MongoDB from Alibaba Cloud’s data centers globally. Figure 1: Data collection and model training data link with MongoDB on Alibaba Cloud. When the vehicle is on the road, the telemetry data is captured through an MQTT gateway, converted into Kafka, and then pushed into MongoDB for data storage and archiving. This data can be used for various applications such as real-time status updates for engine and battery, accident analysis, and regulatory reporting. Figure 2: Mass Production vehicles data link with MongoDB on Alibaba Cloud For a company that is looking to build autonomous driving assistance systems, Alibaba Cloud and ApsaraDB for MongoDB is an excellent technology partner to have. ApsaraDB for MongoDB can handle TBs of diverse sensor data from cars on a daily basis, which doesn't conform to a fixed format. MongoDB provides reliable and highly available data storage for this heterogenous data enabling companies to rapidly expand their system within minutes resulting in time savings when processing and integrating autonomous driving data. By leveraging Alibaba Cloud's ApsaraDB for MongoDB, the R&D team can focus on innovation rather than worrying about data storage and scalability, contributing to faster innovation in the field of autonomous driving. In summary, MongoDB's flexibility, versatility, scalability, real-time capabilities, and strong security framework make it well-suited to address the multifaceted data requirements and challenges that autonomous driving presents. By efficiently managing and analyzing the Big Data generated, MongoDB and Alibaba Cloud are paving the path toward reliable and safe self-driving technology. To learn more about MongoDB’s role in the automotive industry, please visit our manufacturing and automotive webpage .
Building AI with MongoDB: Unlocking Value from Multimodal Data
One of the most powerful capabilities of AI is its ability to learn, interpret, and create from input data of any shape and modality. This could be structured records stored in a database to unstructured text, computer code, video, images, and audio streams. Vector embeddings are one of the key AI enablers in this space. Encoding our data as vector embeddings dramatically expands the ability to work with this multimodal data. We’ve gone from depending on data scientists training highly specialized models just a few years ago to developers today building general-purpose apps incorporating NLP and computer vision. The beauty of vector embeddings is that data that is unstructured and therefore completely opaque to a computer can now have its meaning and structure inferred and represented via these embeddings. Using a vector store such as Atlas Vector Search means we can search and compute unstructured and multimodal data in the same way we’ve always been able to with structured business data. Now we can search for it using natural language, rather than specialized query languages. Considering that 80%+ of the data that enterprises create every day is unstructured, we start to see how vector search combined with LLMs and generative AI opens up new use cases and revenue streams. In this latest round-up of companies building AI with MongoDB, we feature three examples who are doing just that. The future of business data: Unlocking the hidden potential of unstructured data In today's data-driven world, businesses are always searching for ways to extract meaningful insights from the vast amounts of information at their disposal. From improving customer experiences to enhancing employee productivity, the ability to leverage data enables companies to make more informed and strategic decisions. However, most of this valuable data is trapped in complex formats, making it difficult to access and analyze. That's where Unstructured.io comes in. Imagine an innovative tool that can take all of your unstructured data – be it a PDF report, a colorful presentation, or even an image – and transform it into an easily accessible format. This is exactly what Unstructured.io does. They delve deep, pulling out crucial data, and present it in a simple, universally understood JSON format. This makes your data ready to be transformed, stored and searched in powerful databases like MongoDB Atlas Vector Search . What does this mean for your business? It's simple. By automating the data extraction process, you can quickly derive actionable insights, offering enhanced value to your customers and improving operational efficiencies. Unstructured also offers an upcoming image-to-text model. This provides even more flexibility for users to ingest and process nearly any file containing natural language data. And, keep an eye out for notable upgrades in table extraction – yet another step in ensuring you get the most from your data. Unstructured.io isn't just a tool for tech experts. It's for any business aiming to understand their customers better, seeking to innovate, and looking to stay ahead in a competitive landscape. Unstructured’s widespread usage is a testament to its value – with over 1.5 million downloads and adoption by thousands of enterprises and government organizations. Brian Raymond, the founder and CEO of Unstructured.io, perfectly captures this synergy, saying, “As the world’s most widely used natural language ingestion and preprocessing platform, partnering with MongoDB was a natural choice for us. This collaboration allows for even faster development of intelligent applications. Together, we're paving the way businesses harness their data.” MongoDB and Unstructured.io are bridging the gap between data and insights, ensuring businesses are well-equipped to navigate the challenges of the digital age. Whether you’re a seasoned entrepreneur or just starting, it's time to harness the untapped potential of your unstructured data. Visit Unstructured.io to get started with any of their open-source libraries. Or join Unstructured’s community Slack and explore how to seamlessly use your data in conjunction with large language models. Making sense of complex contracts with entity extraction and analysis Catylex is a revolutionary contract analytics solution for any business that needs to extract and optimize contract data. The company’s best-in-class contract AI automatically recognizes thousands of legal and business concepts out-of-the-box, making it easy to get started and quickly generate value. Catylex’s AI models transform wordy, opaque documents into detailed insights revealing rights, obligations, risks, and commitments associated with the business, its suppliers, and customers. The insights generated can be used to accelerate contract review and to feed operational and risk data into core business systems (CLMs, ERPs, etc.) and teams. Documents are processed using Catylex’s proprietary extraction pipeline that uses a combination of various machine learning/NLP techniques (custom Named Entity Recognition, Text Classification) and domain expert augmentation to parse documents into an easy-to-query ontology. This eliminates the need for end users to annotate data or train any custom models. The application is very intuitive and provides easy-to-use controls to Quality Check the system-extracted data, search and query using a combination of text and concepts, and generate visualizations across portfolios. You can try all of this for free by signing up for the “Essentials'' version of Catylex . Catylex leverages a suite of applications and features from the MongoDB Atlas developer data platform . It uses the MongoDB Atlas database to store documents and extract metadata due to its flexible data model and easy-to-scale options, and it uses Atlas Search to provide end users with easy-to-use and efficient text search capabilities. Features like highlighting within Atlas Search add a lot of value and enhance the user experience. Atlas Triggers are used to handle change streams and efficiently relay information to various parts within the Catylex application to make it event-driven and scalable. Catylex is actively evaluating Atlas Vector Search. Bringing together vector search alongside keyword search and database in a single, fully synchronized, and flexible storage layer, accessed by a single API, will simplify development and eliminate technology sprawl. Being part of the MongoDB AI Innovators Program gives Catylex’s engineers direct access to the product management team at MongoDB, helping to share feedback and receive the latest product updates and best practices. The provision of Atlas credits reduces the costs of experimenting with new features. Co-marketing initiatives help build visibility and awareness of the company’s offerings. Harness Generative AI with observed and dark data for customer 360 Dataworkz enables enterprises to harness the power of LLMs with their own proprietary data for customer applications. The company’s products empower businesses to effortlessly develop and implement Retrieval-Augmented Generation (RAG) applications using proprietary data, utilizing either public LLM APIs or privately hosted open-source foundation models. The emergence of hallucinations presents a notable obstacle in the widespread adoption of Gen AI within enterprises. Dataworkz streamlines the implementation of RAG applications enabling Gen AI to reference its origins, consequently enhancing traceability. As a result, users can easily use conversational natural language to produce high-quality, LLM-ready, customer 360 views powering chatbots, Question-Answering systems, and summarization services. Dataworkz provides connectors for a vast array of customer data sources. These include back-office SaaS applications such as CRM, Marketing Automation, and Finance systems. In addition, leading relational and NoSQL databases, cloud object stores, data warehouses, and data lake houses are all supported. Dataflows, aka composable AI-enabled workflows, are a set of steps that users combine and arrange to perform any sort of data transformation – from creating vector embeddings to complex JSON transformations. Users can describe data wrangling tasks in natural language, have LLMs orchestrate the processing of data in any modality, and merge it into a “golden” 360-degree customer view. MongoDB Atlas is used to store the source document chunks for this customer's 360-degree view and Atlas Vector Search is used to index and query the associated vector embeddings. The generation of outputs produced by the customer’s chosen LLM is augmented with similarity search and retrieval powered by Atlas. Public LLMs such as OpenAI and Cohere or privately hosted LLMs such as Databricks Dolly are also available. The integrated experience of the MongoDB Atlas database and Atlas Vector Search simplifies developer workflows. Dataworkz has the freedom and flexibility to meet their customers wherever they run their business with multi-cloud support. For Dataworkz, access to Atlas credits and the MongoDB partner ecosystem are key drivers for becoming part of the AI Innovators program. What's next? If you are building AI-enabled apps on MongoDB, sign up for our AI Innovators Program . We’ve had applicants from all industries building for a huge diversity of new use cases. To get a flavor, take a look at earlier blog posts in this series: Building AI with MongoDB: First Qualifiers includes AI at the network edge for computer vision and augmented reality; risk modeling for public safety; and predictive maintenance paired with Question-Answering systems for maritime operators. Building AI with MongoDB: Compliance to Copilots features AI in healthcare along with intelligent assistants that help product managers specify better products and help sales teams compose emails that convert 2x higher. Finally, check out our MongoDB for Artificial Intelligence resources page for the latest best practices that get you started in turning your idea into AI-driven reality.