Last year, we introduced the MongoDB MCP Server to give agents the connectivity they need to interact with MongoDB, helping them generate context-aware code. But connectivity was only the start. Agents are generalists by design, and they don't inherently know the best practices and design patterns that real-world production systems demand.

Today, we're addressing this by introducing official MongoDB Agent Skills: structured instructions, best practices, and resources that agents can discover and apply to generate more reliable code across the full development lifecycle, from schema design and performance optimization to implementing advanced capabilities like AI retrieval.

To bring this directly into the tools you use, we're also launching plugins for Claude Code, Cursor, Gemini CLI, and VS Code, combining the MongoDB MCP Server and Agent Skills in a single, ready-to-use package.

Turning coding agents into MongoDB experts Coding agents are great at producing working code, but they still make common mistakes in production systems, often defaulting to relational thinking that doesn't translate well to MongoDB, such as:

Over-normalizing schemas, ignoring MongoDB's document-oriented strengths.

Underusing compound indexes, causing performance bottlenecks at scale.

Misusing indexes and search indexes, overlooking the consistency trade-off for high-performance full-text search.

Because these pitfalls mirror common human errors, they are naturally reflected in agent outputs. MongoDB Agent Skills address this by providing expert guidance to agents, like schema design heuristics, indexing strategies, query patterns, and operational safeguards, enabling agents to ship more reliable, more consistent code faster.

Agent Skills were introduced by Anthropic as an open standard and have since been adopted by the leading AI development tools, including Claude Code, Cursor, Codex, and more.

This initial release covers the full application development lifecycle on MongoDB, from connection management and schema design to guidance on implementing advanced capabilities. We will continue to update and expand our skills library based on user needs.

Figure 1. MongoDB Agent Skills.

Scaling agentic engineering with MongoDB As organizations embrace agentic software engineering, existing processes and workflows must be reimagined. The MongoDB MCP Server and MongoDB Agent Skills are built for this shift and work best together, giving builders and agents the tools to move fast without sacrificing guardrails or control.

The MongoDB MCP Server serves as the connectivity layer for your MongoDB deployments. It manages authentication and defines exactly what agents can access and do. Combined with MongoDB’s native authorization, it ensures agents operate with only the permissions they need, while giving teams governance through configurable controls like disabling specific tools.

Agent Skills ensure agents follow best practices from the start, reducing architectural risk, accelerating implementation, and raising the baseline quality of every agent-generated code.

While some skills can be used independently, others work in conjunction with the MongoDB MCP Server for workflows that require it. To simplify setup, the MCP Server and skills are now packaged together as plugins and extensions for Claude Code, Cursor, Gemini CLI, and VS Code, bringing these capabilities directly into your preferred tools.

Figure 2. MongoDB for Claude plugin in action.

We also encourage you to build your own skills as your agentic workflows mature. Whether enforcing internal naming conventions, custom data modeling patterns, or team-specific workflows, skills give you a practical way to codify institutional knowledge and ensure every agent and every developer works from the same playbook.

How to get started Whether you’re using Claude Code, Cursor, Gemini CLI, or other AI development tools, you can install the MongoDB MCP Server and Agent Skills in seconds.

For example, in Claude Code, install the plugin that bundles both:

Code Snippet /plugin marketplace add mongodb/agent-skills /plugin install mongodb@mongodb-plugins

For Cursor, Gemini CLI, and VS Code extensions, refer to their respective documentation.

You can also install the skills for most coding agents using the Vercel Skills CLI (requires Node.js):

Code Snippet npx skills add mongodb/agent-skills

If you prefer, you can manually clone the GitHub repository and copy the skills into the appropriate folder for your agent.

Similarly, to install the MongoDB MCP Server, use the following command:

Code Snippet npx mongodb-mcp-server@latest setup

Agentic engineering is changing how teams work, and it is changing fast. Agents need the context and guidance to meet the standards of real-world production applications. With the official MongoDB Agent Skills and plugins, builders can move faster with confidence, and organizations can adopt coding agents knowing that MongoDB best practices are embedded directly into every workflow.

Next Steps Ship faster, more reliable apps on MongoDB with Agent Skills. Install for Claude Code, Cursor, Gemini CLI and VS Code!

Evolving the developer workflow The modern developer’s workflow is incredibly fast-paced. With developers juggling an average of 14 different tools daily, the cognitive load of constantly jumping between applications can easily disrupt focus. When your application needs to evolve, working with your data shouldn’t force a break in your flow state.

As the MongoDB for VS Code extension has grown to nearly 3 million downloads, we’ve seen firsthand how developers are pushing the boundaries of what an in-IDE (integrated development environment) database tool can do. While developers love accessing their data directly in the editor, we wanted to transform this experience to be even more visual, actionable, and seamless. Instead of switching to external terminals for quick tasks or taking the time to translate familiar MongoDB Shell commands into Extended JSON (EJSON), we are bringing a full-fledged, intuitive data management suite right to your VS Code sidebar.

Exploring what’s new in the MongoDB for VS Code extension Here are the key improvements that transform the extension into a complete workflow solution:

1. Paginated tree view and prescriptive titles Understanding complex data models at a glance is crucial for rapid development. We are transforming the document browsing experience by automatically detecting human-readable fields (like names or emails) to create prescriptive document titles, rather than just displaying standard _id hashes. Furthermore, you can now use a structured, paginated tree view to instantly browse collection data from the “Documents” tab, as well as interactively explore playground results when you run a script. This means you get the full context of your collections visually and instantly.

Figure 1. Paginated tree view and prescriptive titles

2. Powerful action menus and header controls Navigating your data should be inherently actionable. To give you full management capabilities without the need for you to write manual queries, we’ve added a new action header directly inside the tree view. This header equips you with buttons to instantly insert documents, refresh (to rerun the current query or playground script), sort ascending/descending by _id, paginate through results, and even bulk delete to empty a collection.

Additionally, managing individual records is easier than ever. Simply hover over any document within the tree view to reveal a contextual action menu that allows you to instantly delete, copy, clone, and edit the document natively.

Figure 2. Native action menus

3. Native editing and shell syntax default We wanted to make interacting with your database as natural as possible.

To remove the friction of translating your commands, we’ve added a setting that defaults to standard Shell syntax over EJSON for all insert, clone, edit, and clipboard functionalities. This guarantees that any document you copy or any quick fix you make in the extension is instantly compatible with your application code.

Figure 3. Clone action.

Stop context switching and start building Your database tools should adapt to your workflow, not disrupt it. By bringing native data editing, intelligent tree views, and standard Shell syntax directly into your sidebar, we’re bridging the gap between writing code and managing data. You no longer have to sacrifice your flow state just to make a quick database fix, verify a playground result, or translate verbose EJSON formats. This overhaul is another step in our commitment to making this MongoDB extension your ultimate command center—empowering you to spend less time wrestling with external tools and more time actually building your application.

A latency spike hits the dashboard, and metrics signal that the database is under pressure. The cause? Indeterminate. Meanwhile, the business impact is immediate: orders fail to process, customers can’t access accounts, transactions stall, and critical records become temporarily unavailable. Every minute of uncertainty translates into lost revenue, frustrated users, and escalating pressure.

Teams often fall back on a familiar—yet time-consuming—ritual: logging into their data platform, exporting large log files, extracting compressed archives, and manually searching through thousands of lines of entries to identify the issue. What should be a quick diagnosis becomes a manual context-switching investigation. By the time the problematic query, configuration issue, or audit event is identified, users have already experienced the disruption—and the business has absorbed the cost.

MongoDB believes the database should be the heartbeat of a digital business. So we’re introducing a new log integration that brings MongoDB Atlas system and audit logs directly into external observability and storage platforms. This enhancement helps bridge the gap between metrics and meaning when it matters most.

Flexible log delivery for modern observability workflows Now database operators, DevOps pros, and IT Operations teams alike can send MongoDB system and audit logs—including mongod, mongos, and audit logs—directly to the tools they already rely on: Datadog, Splunk, Google Cloud Storage, Azure Blob Storage, or Amazon S3.

Beyond native integrations, MongoDB supports sending logs via OpenTelemetry (OTel), the open-source standard for collecting and transmitting telemetry data. This enables customers to export MongoDB logs to any observability or logging backend that supports OTel. By using a vendor-neutral, standards-based protocol, MongoDB fits seamlessly into modern observability architectures. This eliminates lock-in and preserves flexibility as tooling strategies evolve.

Enabling real-time clarity Modern enterprises generate rich system logs essential for debugging and compliance. However, when these logs are siloed, operational inefficiencies grow. Manual log access introduces friction, delays resolution, and creates a visibility gap between metrics and logs.

MongoDB’s new log integration transforms that experience with:

Accelerated troubleshooting: Send logs in near real-time to observability platforms like Datadog, Splunk, or OpenTelemetry-compatible backends, enabling teams to quickly identify issues and reduce manual operational steps that slow incident resolution.

Unified telemetry: Correlate MongoDB logs with application traces and infrastructure metrics in existing observability platforms, helping teams quickly understand how database behavior impacts overall system performance.

Simplified compliance: Automatically route audit logs to secure long-term storage such as Amazon S3, helping organizations meet regulatory and audit requirements without manual log management.

Figure 1. Atlas Log Integration configuration options for delivering MongoDB logs to observability and storage platforms.

image

Real-world use cases How does this look in practice for modern application, operations, and engineering teams? Here are a few examples.

table

The criticality of observability As applications scale, the database becomes the most critical layer of an organization’s technology stack. Missing or siloed visibility leads to costly downtime and fragmented decision-making.

This log integration is available for dedicated M10+ clusters. An external sink can be configured in minutes:

Navigate to the Project Integrations page in the MongoDB Atlas UI.

Select the intended destination: Datadog, Splunk, Google Cloud, Microsoft Azure, Amazon S3, or any OTel log endpoint.

Enter the required credentials and select the desired logs to send: mongod, mongos, or audit.

Note: Atlas Search logs are also currently available via private preview.

Figure 2. MongoDB Atlas logs integrated into an OpenTelemetry observability pipeline.

image

One observability strategy, built to scale For teams that need fast, MongoDB-centric visibility, MongoDB Atlas continues to offer powerful native tools like Query Insights and the Query Profiler. These capabilities are designed to surface what is happening inside a user’s clusters with minimal friction.

However, as organizations scale, database insights can not live in isolation. MongoDB Atlas’s log integration extends observability systematically to the data plane. This enables MongoDB logs to flow into the observability platforms teams already use across engineering, security, IT operations, and compliance. With native integrations and an OpenTelemetry-compatible endpoint, teams can route logs wherever they are needed. This enables rapid troubleshooting, stronger auditability, and confident scaling without blind spots.

Image of a wall in the MongoDB Dublin office that is painted with Dublin relevant illustrations and text that says "Build together" and "Make it matter" As MongoDB’s international headquarters, Dublin is a key hub where over 300 employees from more than 40 nationalities own critical parts of the company’s products and support customers running mission-critical systems across the globe. Established in 2012, MongoDB Dublin has long played a pivotal role in helping the company achieve its mission of empowering innovators to create, transform, and disrupt industries by unleashing the power of software and data.

In this spotlight, you’ll hear from people across MongoDB’s Product & Technology, Sales, and Technical Services teams about what it’s like to build your career—and your life—in Dublin with MongoDB.

Image of CEO, CJ Desai, speaking in front of a group of employees in the Dublin office. CEO CJ Desai holds an “Ask Me Anything” session in a recent visit to Dublin. Life at MongoDB Dublin The MongoDB Dublin office represents both the company’s global energy and the warmth of Irish culture. Ciara, workplace manager for the office, notes:

There’s always something happening here, from fun in-office activities and lively trivia nights to vibrant town halls that bring people together. We run regular volunteer days that give everyone the chance to give back, along with coffee mornings, team socials, and plenty more that keep the atmosphere welcoming and connected.

There’s a real sense that MongoDB Dublin captures true Irish charm and character, friendly, open, and full of personality. That mix creates a workplace that feels both globally diverse and locally grounded. You get the best of both worlds: international perspectives combined with that uniquely Irish sense of humour and hospitality.

MongoDB Dublin isn’t just a workplace—it’s a team that wins and grows together!

Here’s what stands out about life and work here:

Wellness comes first: From private health, dental, and vision care, to Gympass discounts and Headspace subscriptions, we support your physical and mental well-being.

Family and flexibility: Eligible employees enjoy 20 weeks of paid parental leave, fertility and family planning support through Carrot, and Cleo resources for parenting. Our flexible work model means you never have to choose between career and family.

Financial security: We offer life insurance, income protection, and the chance to grow wealth through our employee stock purchase plan.

Time for you: Every employee enjoys 27 days of annual leave, plus extra wellness initiatives like MongoDB Bloom (our internal wellness program), midday meditation sessions, and fun in-office perks.

Growth and development: You’ll have access to dedicated learning programs, technical training and leadership development to upskill in programming, leadership, communication, and more.

Inclusion and community: With over 40 nationalities represented in our Dublin office, our teams reflect the global customers and communities we serve. From active Employee Resource Groups (ERGs) like MDBWomen, Queer Collective, and Config to a warm, collaborative office culture, MongoDB Dublin is a place where you’ll feel supported—whether you’re relocating to Ireland or continuing to grow your career here.

2 separate images of colleagues from MongoDB smiling together during volunteer opportunities. Local volunteer opportunities, team outings, and other events to bring colleagues together Hear from the Dublin team Donal, product & technology leader for EMEA, describes how MongoDB’s platform and culture drew him in:

Joining MongoDB felt like the right move at the right time. MongoDB is an amazing platform, with capabilities ranging from deploy anywhere, support for unstructured and semi-structured data, native vector search for embeddings, and flexible workload handling. The problems we’re tackling around data, scale, and flexibility are real, hard problems and they matter to how modern applications are built.

What’s really impressed me since joining is the engineering culture. People are curious, thoughtful, and open to learning, bringing strong ownership and intellectual honesty to their work. Building out our engineering teams in Ireland is exciting because we’re creating groups that will own critical parts of the product, not just contributing at the edges.

It’s a great moment to join MongoDB in Ireland. The company is growing, the Dublin office is becoming a core engineering hub, and engineers here have the chance to make a real impact. If you’re looking for challenging work, smart teammates, and the opportunity to help shape what we’re building, this is a really exciting place to be.

2 separate images of multiple employees of MongoDB attending events in the Dublin office. You’ll find the office frequently buzzing with events. For Max, an engineer working on query optimization, joining MongoDB Dublin has been both a professional challenge and a personal milestone:

MongoDB was the first database I ever used when learning how to build backend applications, so getting the opportunity to intern here and now work here full-time feels like a real full-circle moment. Today, I work on query optimization, a core part of the database engine focused on making queries as fast as possible. It brings a lot of interesting theoretical computer science problems. We get to dive into research papers and actually implement those ideas in practice, which means I’m learning something new every day.

The Dublin team is incredibly international, with talented engineers from all over the world. I genuinely feel lucky to work alongside people with such deep experience building databases. Especially at the start of my career, being surrounded by colleagues I look up to makes a huge difference. We’re often tackling hard problems that haven’t been solved before in the context of document databases, so having that level of expertise around you is invaluable.

It’s been almost a year since I moved to Dublin, and it’s flown by. Relocating to a new country is always a bit daunting, but Ireland is such a welcoming place that it already feels like a second home. Dublin doesn’t always get the credit it deserves, but being so close to both the sea and mountains is something I really value. What more could you ask for?

An image of 3 employees working and taking meeting calls from within small private phone booths. Need a space for focus time? Grab a phone booth! On the sales side, Dublin is at the crossroads of cloud, AI, and data-driven innovation, making it an exciting place to build a long-term career in tech sales.

Account executive Stephane shares how MongoDB’s culture and the Dublin market shape the experience:

Selling at MongoDB stands out because of the sales culture and investment in people. You’re constantly challenged to adapt, stay flexible (much like the document model), and raise the bar, both individually and as a team. There’s a strong entrepreneurial mindset here. If you’re curious, proactive, and eager to learn, you’re given real ownership and the support to run with your ideas, rather than being tied to a one-size-fits-all script. It’s why Think Big, Go Far genuinely shows up in how we work.

Dublin sits at the centre of cloud, AI, and data-driven innovation, making it a great place to build a long-term career. Here, you’re not just selling software - you’re helping customers modernise mission-critical systems and move from experimentation to real production impact. I joined MongoDB because I believed in the impact of cloud and digital transformation, and that opportunity has only grown. Now, with AI driving the next wave of innovation, MongoDB is well-positioned to play a meaningful role, and it’s exciting to be part of that journey.

What really sums up sales at MongoDB is the opportunity to grow. There’s a clear path forward, whether that’s moving from sales development representative to account executive, stepping into leadership, or exploring new roles across the business. If you’re motivated, open to learning, and serious about building a long-term sales career, MongoDB gives you the environment and support to do it.

The image on the left shows employees taking a break by playing foosball and billiards together. The image on the right shows employees gathered at a table with open laptops and a larger screen with a presentation. Silvia, an account development representative, recalls how the culture came through even before she joined—and how it’s shown up since:

From the very first interviews, I was struck by how clearly MongoDB’s values came through. Every conversation felt supportive and intentional. What stood out most was the genuine investment in me as a candidate, making sure I had what I needed to succeed before I’d even contributed anything back.

Once you join, that support only grows. It’s a high-performance environment and the work is challenging, but you’re never doing it alone. Learning is constant, collaboration is open, and from week one, you feel like an active part of the team, without rigid hierarchies holding you back.

Walking into the Dublin office for the first time, I immediately felt the sense of community. In my first month, I met both our CIO and CEO in person. Hearing them speak about the company’s ambition made it clear this is something bigger, and that the work we do here truly matters.

An image of two employees smiling with #LifeAtMongoDB photo props. The annual summer party is always good craic. The technical services team in Dublin plays a critical role in helping customers succeed on MongoDB. For Fernanda, a manager on the team, that journey has been about impact, community, and the confidence to grow her career in a new country:

I joined MongoDB in 2018, when the Cloud support team had just four people across all of EMEA. From the start, there was a strong culture of ‘we’re here to help each other, you can count on me.’ It made us much more than colleagues, and I had never experienced such a supportive environment before.

What’s kept me here is that this culture never faded as we scaled. Even on cold, rainy Dublin mornings, there’s real warmth in the office, smart conversations, and a real sense that you’re not facing challenges alone. That combination of impact, continuous learning, and a true ‘team first’ mindset is what’s made me stay and grow my career here.

Today, Dublin is larger, more diverse, and deeply connected to global initiatives, but the essence remains the same. You feel welcomed, people are approachable, and there’s a real Build Together mindset across teams.

Ready to build your career in Dublin? Whether you’re building core database features or partnering with customers on their most strategic initiatives, at MongoDB Dublin, you’ll be surrounded by people who want to see you succeed and by teams that are shaping the future of software.

As mentioned in the previous post, a deeper exploration of the concepts covered in this post is covered in our recently published VLDB ’25 paper, Design and Modular Verification of Distributed Transactions in MongoDB.

Modular, Model-Based Verification As discussed in Part 1, we had developed a TLA+ specification of MongoDB’s distributed transactions protocol in a compositional manner, describing the high level protocol behavior while also formalizing the boundary between the distributed aspect of the transactions protocol and the underlying single-node WiredTiger storage engine component. As mentioned, the distributed transactions protocol can be viewed as running atop the lower level storage layer.

When considering the correctness guarantees of the distributed transactions protocol, a subtle aspect is its interaction with the concurrency control mechanisms at each layer of the system. In particular, its interaction with the WiredTiger storage engine API, which has various timestamp-based operations implemented to adequately support the correct operation of the distributed transactions protocol.

Our formal model of the distributed transactions protocol was useful for checking its guarantees, but we also wanted to formalize the contract between this distributed protocol and the underlying storage engine, to check that this interface boundary, as defined in our abstract specification, was matched by the implementation. Leveraging the clean interface boundary in our specification, we developed a tool for automatically checking conformance between the WiredTiger implementation and this abstract storage specification (Storage) we defined. Our storage layer specification itself serves as a useful and precise definition of a subset of the WiredTiger storage engine semantics, but we can also use it for automatically generating test cases to check conformance of WiredTiger semantics to our spec.

Figure 1. Compositional specification of distributed transactions in TLA+ and storage engine component.

Path-based test case generation To do this, we make use of a modified version of the TLC model checker to first generate the storage component specification’s complete graph of reachable states for finite parameters. We then compute a set of path coverings in this graph, where each path is then converted to an individual test case as a sequence of underlying storage engine API calls. This model-based verification technique allows us to automatically generate tens of thousands of individual test cases for WiredTiger which each check that the implementation matches the behavior defined in our abstract specification, which is also the contract relied on by the high level, distributed transactions protocol.

Figure 2. Model-based test case generation workflow.

For a small, complete finite model (2 keys and 2 transactions), we are able to check conformance with WiredTiger with a generated suite of 87,143 tests, which are generated and executed against the storage engine in around ~40 minutes.

Figure 3. Test case generation statistics for the storage layer model.

Our current storage layer specification can be found in Github, and a link to an interactive, explorable version of it can also be found in Github. In the future, we hope to explore modeling of a more extensive subset of the WiredTiger API and explore alternate state space exploration strategies for generating tests, e.g., randomized path sampling, or other strategies that only require approximate coverage of the state space model. This approach bears similarity to other testing efforts we had explored in the past.

Conclusion Modeling our distribution transactions protocol enabled verification of protocol correctness, and our use of modularity and compositional verification was instrumental in letting us reason about these high-level correctness properties while also being able to automatically check that our abstract storage interface correctly matches the semantics of the implementation. Similar approaches have been explored recently for model-based verification of distributed systems, as well as other approaches for testing systems using path-based test case generation techniques. In the future, we also remain optimistic about the role that LLMs can play in this type of verification workflow. For example, once such a testing harness for checking conformance is developed, having LLMs autonomously develop a model of the underlying system behavior allows for precise characterization of behavior, along with automated test suite generation capabilities.

You can find more details about this work in our recently published VLDB ‘25 paper, and in the associated Github repo that contains our specifications and code, as well as links to in-browser, explorable versions of our specifications.

For example, the energy at January’s MongoDB.local San Francisco—where we announced capabilities to help teams ship production AI faster—was infectious. MongoDB isn’t just starting a new chapter in AI; we’re rewriting the book in real time.

The next generation of AI companies isn't just looking for a temporary place to store data; they’re looking to build on a generational modern data platform. Indeed, the most innovative founders are moving away from rigid, legacy systems and embracing a single, fluid foundation that can grow with them.

At MongoDB.local SF, our message was clear: Choose your data platform strategically in order to ship faster. From our new Voyage 4 models to the general availability of our Intelligent Assistant, we are obsessed with anticipating what developers need next. This assistant is particularly impactful because it embeds MongoDB-specific expertise directly into Compass and MongoDB Atlas, allowing developers to troubleshoot performance without the "context-switching" that traditionally slows them down.

In this issue, I’m thrilled to spotlight four startups who are building the future on the right foundation. You’ll see how Modelence and Thesys are using our flexible document model to eliminate 'operational drag,' allowing them to iterate on AI-native workflows in real time. And then there’s Heidi and Emergent Labs, who both are proving that when you simplify your codebase with a unified platform, you can turn a plan into shipped code at record speeds.

I’ve highlighted their journeys below so you can see exactly how these leaders are setting a new pace and changing their trajectory with MongoDB.

Modelence Modelence aims to modernize backend infrastructure for the era of AI-assisted development. Traditional relational databases and manual systems create significant operational drag, as their rigid schemas and heavy migrations cannot keep pace with agent-native workflows. These legacy systems struggle with the high-velocity requirements of intelligent coding agents, which must iterate on data structures in real time without causing system downtime.

To build a stable foundation for automation, Modelence integrated MongoDB Atlas as its core data layer. The platform utilizes the flexible document model to align with how intelligent systems think, allowing specifications and runtime events to coexist. This "fit" enables per-tenant isolation and managed credentials, ensuring automated changes remain safe and traceable without the tangle of relational joins.

Standardizing on MongoDB Atlas helped Modelence raise $3 million dollars in its Seed round. The company now moves from planning to running features in minutes, achieving faster iteration loops and fewer regressions.

Thesys Thesys aims to empower developers by making generative user interfaces—adaptive, real-time components—accessible to everyone. Previously, developers faced the friction of static chat bubbles and hardcoded dashboards that failed to visually represent complex AI outputs. These traditional interfaces forced teams to rebuild UI layers for every use case, which kills user engagement.

To solve these orchestration challenges, Thesys integrated MongoDB Atlas as the operational backbone for its C1 API middleware. The platform utilizes the document model to manage complex entities within a single, high-performance data layer. By removing the friction of mapping unstructured LLM outputs to rigid schemas, engineering teams can now ship updates weekly.

Through the MongoDB for Startups program, Thesys successfully accelerated its go-to-market timeline. By offloading operational management to MongoDB Atlas, Thesys now maintains the agility to evolve its data layer alongside emerging AI trends, ensuring its intelligent interfaces remain high-performing as they scale globally.

Emergent Labs Emergent Labs sought to democratize software development through “vibe coding,” a platform where AI agents build applications from natural language prompts. The company’s initial use of PostgreSQL caused significant friction, as AI agents frequently failed during schema migrations when non-technical users iteratively changed their application requirements.

By switching to MongoDB Atlas, Emergent Labs provided its agents with a flexible, document-based architecture that matches the JSON data they naturally produce. This eliminated the PostgreSQL migration loops, allowing agents to modify data structures on the fly and deploy isolated, production-ready databases in minutes.

The transition has powered the creation of nearly 2 million applications across 180 countries in just four months. With MongoDB Atlas, the platform now supports complex builds of up to 300,000 lines of code, doubling deployment rates and allowing non-technical entrepreneurs to launch sophisticated tools without traditional engineering resources.

Heidi Heidi aims to reclaim clinician time by automating administrative tasks. Previously, clinicians spent 40% of their shifts on paperwork, reducing time for patient care. To manage this at scale, Heidi initially used Amazon DocumentDB, but faced critical limitations including mandatory downtime for scaling, high latency, and a lack of native search functionalities essential for complex AI workloads.

To eliminate these bottlenecks, Heidi migrated to MongoDB Atlas for its flexible schema and built-in AI capabilities. Integrating MongoDB Vector Search enables Heidi to perform RAG without "bolt-on" databases, streamlining vector and semantic search under a single API. This technical fit enables developers to unify diverse medical data while meeting stringent healthcare security and regulatory requirements.

Since migrating, Heidi has supported 81 million consultations, returning 18 million hours to the frontline. By offloading management to MongoDB Atlas, Heidi ensures its platform remains high-performing while empowering practitioners to focus on their primary mission: providing compassionate patient care.

Video Spotlight Before you go, watch TinyFish Co-founder and CEO, Sudheesh Nair, explain how “nano agents” are transforming web-based research.

Learn how TinyFish extracts actionable intelligence from unstructured internet data using MongoDB and Voyage AI.

Traditional movie search relies on filtering by genre, actor, or title. But what if you could search by how you feel? Imagine typing:

"something uplifting after a rough day at work"

"a movie that will make me cry"

"I need adrenaline, can't sleep anyway"

"something to watch with grandma who hates violence"

This is mood-based semantic search: matching your emotional state to movie plot descriptions using AI embeddings.

In this tutorial, you will build a mood-based movie recommendation engine using three powerful technologies: voyage-4-nano (a state-of-the-art open-source embedding model), Hugging Face (for model and dataset hosting), and MongoDB Atlas Vector Search (for storing and querying embeddings at scale).

Why mood-based search? Genre tags are coarse. A "drama" can be heartwarming or devastating. A "comedy" can be light escapism or dark satire. Traditional filters cannot capture these nuances.

Semantic search solves this by understanding meaning. When you search for "feel-good movie for a rainy Sunday", the system doesn't look for those exact words. It understands the intent and matches it against plot descriptions that evoke similar feelings.

Architecture overview The system combines three components from the Hugging Face ecosystem with MongoDB:

voyage-4-nano (Hugging Face Hub): Converts text to embeddings (up to 2048 dimensions, we use 1024)

MongoDB/embedded_movies(Hugging Face Datasets): 1500+ movies with plot summaries, genres, cast

MongoDB Atlas Vector Search: Stores embeddings and performs similarity search

Understanding voyage-4-nano voyage-4-nano is the smallest model in Voyage AI's latest embedding series, released with open-weights under the Apache 2.0 license. Voyage AI was acquired by MongoDB, and the Voyage 4 series models are now available through MongoDB Atlas. All models in the series (voyage-4-large, voyage-4, voyage-4-lite, and voyage-4-nano) produce compatible embeddings in a shared embedding space, allowing you to mix and match models within a single use case.

Although voyage-4-nano natively supports embeddings up to 2048 dimensions, we deliberately truncate them to 1024 dimensions using its Matryoshka embedding property. In practice, this provides a strong balance between semantic quality, storage efficiency, and vector search latency, while preserving stable ranking behavior.

Sentence Transformers This tutorial uses Sentence Transformers, a Python library built on top of Hugging Face Transformers. It is specifically designed for working with embedding models and provides a simple API for generating text embeddings.

Why Sentence Transformers instead of raw Transformers? When working with embedding models, you need to handle tokenization, pooling, normalization, and prompt formatting. Sentence Transformers does all of this automatically in a single method call. The code is cleaner, there are fewer potential errors, and you get built-in features like batch processing with progress bars.

Under the hood, Sentence Transformers still uses Hugging Face Transformers to load and run the model.

Configure the development environment Let's get started!

Create the Project Structure Code Snippet 1 mkdir mood-movie-search 2 cd mood-movie-search 3 mkdir src 4 touch requirements.txt .env

Install dependencies Create the requirements.txt file:

Code Snippet 1 cat < requirements.txt 2 fastapi>=0.109.0 3 uvicorn>=0.27.0 4 pymongo>=4.6.1 5 sentence-transformers>=3.0.0 6 python-dotenv>=1.0.0 7 datasets>=2.16.0 8 torch 9 EOF

Create a Python virtual environment and install dependencies:

to be continued...

At MongoDB, we are committed to helping organizations build intelligent applications that span cloud and edge environments seamlessly. That’s why we are excited to highlight our work with ObjectBox, a lightweight, high-performance on-device database and sync solution purpose-built for edge AI and offline-first applications.

Together, MongoDB and ObjectBox are making it easier for developers to build hybrid architectures that deliver fast, private, and resilient AI experiences across devices and environments.

Figure 1. Example cloud-edge AI setup.

Example cloud-edge AI setup. ObjectBox: A purpose-built database for the edge Founded by Markus Junginger and Dr. Vivien Dollinger, ObjectBox was designed specifically to support edge computing and offline-first use cases. At its core, ObjectBox’s design prioritizes efficiency (including speed, privacy, battery use, and memory consumption) and ease of development.

This strong foundation makes ObjectBox particularly well-suited for next-generation applications that need to run reliably in edge environments—whether on a factory floor, in a retail store, or through a remote healthcare device. ObjectBox empowers developers to build responsive, privacy-conscious applications that work even when connectivity is limited or unavailable.

The platform includes the following features:

A fast, local vector database that stores data directly on devices, supporting on-device AI and local vector search.

Built-in data sync, which keeps data consistent across devices even when offline, and now integrates directly with MongoDB.

Multi-language support, including support for C++, Swift, Flutter, Python, Go, Java, and Kotlin, makes ObjectBox accessible to developers across ecosystems.

These features make ObjectBox an ideal solution for building intelligent applications that run reliably at the edge. This includes a wide range of devices—from smartphones and industrial sensors to automotive ECUs and point-of-sale (POS) devices.

Edge to cloud data sync: The MongoDB Atlas native connector ObjectBox's new MongoDB Sync Connector combines local-first edge processing with centralized cloud intelligence (i.e., hybrid AI).

This is increasingly important as organizations seek to process data closer to where it is generated—at the edge—while still benefiting from the power and scalability of the cloud. Managing this dual environment efficiently is key to unlocking performance, resilience, and real-time insights.

Developers can now use ObjectBox for real-time, low-latency operations on edge devices while syncing relevant data to MongoDB Atlas, enabling organizations to achieve:

Long-term storage

Centralized dashboards and analytics

AI model retraining

Cloud-based coordination and automation

This hybrid architecture aligns with how modern applications are being built—distributing intelligence where it makes the most sense.

Figure 2. Central Sync for ObjectBox and MongoDB Atlas.

Central Sync for ObjectBox and MongoDB Atlas. Figure 3. Edge setup for ObjectBox and MongoDB Atlas.

Edge setup for ObjectBox and MongoDB Atlas Bringing AI to the edge isn’t just about performance. It is also about privacy, sustainability, and user experience. By processing data locally:

Privacy is enhanced—sensitive information stays on the device.

Latency is reduced—actions can be taken instantly.

Bandwidth usage drops—lowering costs and improving efficiency.

Battery and CPU use are optimized—extending the life of edge devices.

This aligns with MongoDB’s commitment to empowering developers to build intelligent, resilient, and user-centric applications—wherever they need.

Real-world use cases Industrial IoT Industrial IoT (IIoT) is a prime example of where edge and cloud must work together. On a modern factory floor, everything from low-frequency brownfield devices to high-frequency greenfield machines generates vast amounts of data.

Data generated include vibration levels, temperature readings, pressure changes, and machine runtimes. In short, the sort of data that often needs to be processed locally to monitor systems in real time and to trigger alerts when anomalies or threshold breaches occur.

With ObjectBox running on device, this critical operational data can be captured, analyzed, and used onsite and within AI applications immediately, even with limited or no connectivity. ObjectBox is designed for efficient, high-throughput I/O, enabling real-time processing of high-frequency data streams even on resource-constrained edge devices.

It supports a broad range of data types—from objects and time series data, to tree structures (e.g., UMATI) and vector embeddings—with a lightweight database (typically only a few MB in size). This makes it well-suited for production deployments that need to integrate modern AI and edge workloads with legacy systems and heterogeneous hardware, typical for the manufacturing industry.

The ObjectBox Sync Server can run on almost any device, enabling fast, reliable, and secure offline data synchronization across the shop floor. Paired with the MongoDB Sync Connector, the most relevant insights can then be synced to the cloud, where they can be aggregated, enriched with AI models, and stored for long-term analysis (like anomaly detection or RUL models).

This hybrid architecture enables advanced use cases such as predictive maintenance, where historical records, live equipment data, and machine learning models are combined to forecast potential failures before they happen. (For more details, explore our Predictive Maintenance solutions library.)

With this architecture, the system provides:

Real-time responsiveness on the shop floor

Centralized analytics and cross-site dashboards at cloud scale

Support for predictive maintenance workflows in offline or intermittently connected environments

Unified data access across heterogeneous data sources, from individual sensors to full production lines

By combining low-latency edge processing with centralized intelligence, developers and operators gain visibility into how equipment is performing from the health of a single machine, to trends across an entire fleet or factory network—without compromising performance or reliability.

Figure 4. Industrial IoT.

Industrial IoT. Point-of-sale systems Point-of-sale (POS) systems—such as those used in restaurants—are another strong fit for edge AI and hybrid architectures. During peak dining hours, cashiers and servers need instant, reliable access to menus, order histories, and payment processing—even if their internet connection is unstable or drops.

With ObjectBox’s offline-first, on-device database, restaurants can process real-time transactions, track inventory, and personalize customer experiences with local AI directly at the POS terminal. This helps store owners avoid service disruptions or lost sales.

With MongoDB Sync Connector, relevant data (like sales trends, customer preferences, and stock levels) syncs to MongoDB Atlas. This enables restaurant managers to run centralized dashboards, perform demand forecasting, and train AI models that optimize staffing, menu design, and supply chain planning.

In summary, this hybrid POS architecture of local-first responsiveness and cloud-powered insights ensures:

Seamless customer experiences without downtime

Accurate, up-to-date data whenever needed

Scalable, resilient operations across multiple restaurant locations

Figure 5. Point-of-sale systems.

Point-of-sale systems. What’s next With the release of ObjectBox 5.0 and its new MongoDB Connector, ObjectBox has taken a major step toward simplifying user‑specific data sync at the edge. Together, MongoDB and ObjectBox offer a modern foundation for building intelligent, distributed applications that run reliably from device to cloud. This partnership makes it easier than ever to pair low‑latency edge data processing with the flexibility, security, and global reach of MongoDB Atlas.

Building AI applications means solving real problems: keeping conversational context clean and queryable, retrieving the right information from thousands of past interactions, connecting AI agents to your data without custom plumbing. These aren't theoretical challenges, they're the friction points that slow teams down every day.

The AI era demands more from your data platform. MongoDB gives you everything you need to build quickly.

Voyage AI: the best gets better Embedding models can make or break AI search experiences. We're proud that voyage-3-large has been the world's top-performing embedding model on Hugging Face's RTEB benchmark since its inception.

But we didn’t rest on our laurels. There’s a new model at the top of the charts.

Today, we're pleased to announce that the Voyage 4 model family is now generally available. The best just got better. The voyage-4 series models operate in a shared embedding space, allowing for cross-model compatibility and unprecedented flexibility to optimize for accuracy, speed, or cost. This release also includes voyage-4-nano, our first open-weight model available on HuggingFace, perfect for local development.

Additionally, we're launching the new voyage-multimodal-3.5 model, which has been specifically trained to support video content alongside text and images. For developers building multimodal AI applications, this represents a significant leap forward in handling diverse content types within a single retrieval system. Best of all, upgrading is remarkably straightforward—you can simply change the model parameter to "voyage-multimodal-3.5" in your API call, instantly unlocking video capabilities without needing to refactor your existing codebase or change your application architecture.

Finally, we’re announcing the public preview of the Embedding and Reranking API on MongoDB Atlas, providing API support for Voyage AI models. While enabling standalone usage of the models with any technology stack, the API benefits from the robust security and scalability standards of MongoDB. By bringing critical components into a single control plane and interface, it eliminates the need to manage separate vendors and significantly reduces operational overhead.

Automated Embedding, convenience built into MongoDB Community Persistence matters. An AI with amnesia isn’t helpful; users need systems to remember context from minutes, hours, and weeks ago. Every interaction is a goldmine of preferences, patterns, and behavior that should make the next interaction smarter.

But storing conversation history in a database isn't enough. Simple storage solves nothing if you can't retrieve the right information at the right time. The real challenge is intelligent retrieval: finding relevant context across thousands of past interactions, filtered by metadata and user attributes, without your system buckling under production load. This is where vector search becomes critical—enabling semantic search that captures meaning, not just keywords, while operating on your real-time operational data. And this is where MongoDB's approach eliminates a major pain point: the need to sync data between separate systems for vectors and application data.

Until now, generating and storing these vectors required overhead—development time, infrastructure management, and cognitive load. No longer.

We're introducing Automated Embedding for MongoDB Community Edition in public preview. MongoDB Community Edition now handles the complexity of managing embedding models automatically, giving developers high-accuracy semantic search in the database while maintaining flexibility to use any LLM provider or orchestration framework. Automated Embedding offers one-click automatic embedding directly inside MongoDB, which eliminates the need to sync data and manage external models. It’s an easy way to get high quality embedding natively.

Best-in-class retrieval shouldn't require infrastructure work—Automated Embedding in MongoDB Vector Search delivers on that promise. Automated Embedding in MongoDB Vector Search is available now in Community Edition, with Atlas access coming soon.

Precise text filtering for advanced search use cases Today, we announced the launch of Lexical Prefilters for Vector Search. This addresses a long-standing request from developers building semantic search interfaces who need advanced text filtering alongside vector operations.

The new syntax enables powerful text filtering capabilities—fuzzy matching, phrase search, wildcards, and geospatial filtering—as prefilters for vector search. This leverages full text analysis capabilities while maintaining the semantic power of vector search. We've introduced a new vector data type in $search index definitions and a vectorSearch operator within the $search aggregation stage to make this work seamlessly.

This replaces the knnBeta operator with a cleaner, more powerful approach. For teams already using lexical and vector search together, this provides a simplified migration path with significantly expanded capabilities.

Intelligent assistance wherever you work MongoDB’s intelligent assistant is generally available in MongoDB Compass. The assistant provides in-app guidance for debugging connection errors, optimizing query performance, and learning best practices, all without leaving your development environment. You can even query your database using natural language through read-only database tools that require your approval before execution, allowing for deeper contextual awareness of your data.

The assistant was built to address real friction: developers switching between multiple tools and documentation tabs, waiting for support responses, or getting generic advice from general-purpose AI chatbots that don't understand MongoDB-specific contexts. Now, tailored guidance is available instantly, right where you're working.

The modernized Atlas Data Explorer interface brings the Compass experience directly into the Atlas web UI, addressing a critical gap for teams with security policies that restrict desktop application usage. Users can now perform sophisticated query development, optimization, bulk operations, and complex aggregations—all with AI assistance—across all MongoDB Atlas clusters in a unified web interface.

Whether you're troubleshooting a connection issue, optimizing a slow query, or learning how to structure an aggregation pipeline, the intelligent assistant delivers MongoDB-specific expertise without context switching. Try the intelligent assistant in the modernized Atlas Data Explorer now.

The engine behind MongoDB Search and Vector Search is now available under SSPL Finally, mongot, the engine powering MongoDB Search and Vector Search, is now publicly available under SSPL. While still in preview, after years of development and investment, we're making the source code of this core technology available to the community, expanding our unified search architecture beyond Atlas to every MongoDB deployment.

mongot runs separately from mongod, MongoDB's core database process, and is the foundation that makes powerful search native to MongoDB. Releasing mongot under SSPL means full transparency for security audits and debugging complex edge cases. Developers can dive into mongot's architecture, understand how search and vector operations work under the hood, and help shape the future of search at MongoDB.

A modern data platform that evolves with your needs These announcements reflect our commitment to anticipating what developers need as AI development matures. Vector search, time series, stream processing, queryable encryption, Atlas itself—we've consistently delivered on emerging requirements. "If you're building an early-stage company that is going to scale very rapidly, you need a database solution that isn't going to break under the load of a huge volume of users," said Eno Reyes, Co-founder and CTO of Factory. "You need a fast-moving team with a reliable solution, and there really is one option in this space—and it's MongoDB."

Rabi Shanker Guha, CEO of Thesys, put it this way: “MongoDB helps us move fast in an ever-changing world. The best database is the one you don’t have to think about—it just works exactly where and how you need it. That’s MongoDB for us.”

Ship faster, scale confidently Each capability we announced today addresses real friction in the AI development workflow and in the developer experience. We're not asking developers to choose between structured data and vectors, between performance and flexibility, or between rapid iteration and production readiness.

The promise is straightforward: ship faster, scale confidently, and focus on what makes your AI application unique—not on managing database infrastructure. In an ecosystem crowded with point solutions and retrofitted legacy systems, MongoDB is a modern data platform built for the long haul.

Vision RAG makes complex documents—including their figures and tables—searchable by using multimodal embeddings, eliminating the need for complex and costly text extraction. This guide explores how Voyage AI’s latest model powers this capability and provides a step-by-step implementation walkthrough.

Vision RAG: Building upon text RAG Vision RAG is an evolution of traditional RAG built on the same two components: retrieval and generation.

In traditional RAG, unstructured text data is indexed for semantic search. At query time, the system retrieves relevant documents or chunks and appends them to the user’s prompt so the large language model (LLM) can produce more grounded, context-aware answers.

Figure 1. Text RAG with Voyage AI and MongoDB.

Text RAG with Voyage AI and MongoDB Enterprise data, however, is rarely just clean plain text. Critical information often lives in PDFs, slides, diagrams, dashboards, and other visual formats. Today, this is typically handled by parsing tools and OCR services. Those approaches create several problems:

Significant engineering effort to handle many file types, layouts, and edge cases

Accuracy issues across different OCR or parsing setups

High costs when scaled across large document collections

Next-generation multimodal embedding models provide a simpler and more cost-effective alternative. They can ingest not only text but also images or screenshots of complex document layouts, and generate vector representations that capture the meaning and structure of that content.

Vision RAG uses these multimodal embeddings to index entire documents, slides, and images directly, even when they contain interleaved text and images. This enables them to be searchable via vector search without requiring heavy parsing or OCR. At query time, the system retrieves the most relevant visual assets and feeds them, along with the text prompt, into a vision-capable LLM to inform its answer.

Figure 2. Vision RAG with Voyage AI and MongoDB.

Vision RAG with Voyage AI and MongoDB As a result, vision RAG enables LLM-based systems with native access to rich, multimodal enterprise data, while reducing engineering complexity and avoiding the performance and cost pitfalls associated with traditional text-focused preprocessing pipelines.

Voyage AI’s latest multimodal embedding model The multimodal embedding model is where the magic happens. Historically, building such a system was challenging due to the modality gap. Early multimodal embedding models, such as contrastive language-image pretraining (CLIP)-based models, processed text and images using separate encoders. Because the outputs were generated independently, results were often biased toward one modality, making retrieval across mixed content unreliable. These models also struggled to handle interleaved text and images, a critical limitation for vision RAG in real-world environments.

Voyage-multimodal-3 adopts an architecture similar to modern vision-capable LLMs. It uses a single encoder for both text and visual inputs, closing the modality gap and producing unified representations. This ensures that textual and visual features are treated consistently and accurately within the same vector space.

Figure 3. CLIP-based architecture vs. voyage-multimodal-3’s architecture.

CLIP-based architecture vs. voyage-multimodal-3’s architecture This architectural shift enables true multimodal retrieval, making vision RAG a viable and efficient solution. For more details, refer to the voyage-multimodal-3 blog announcement.

Implementation of vision RAG Let’s take a simple example and showcase how to implement vision RAG. Traditional text-based RAG often struggles with complex documents, such as slide decks, financial reports, or technical papers, where critical information is often locked inside charts, diagrams, and figures.

By using Voyage AI’s multimodal embedding models alongside Anthropic’s vision-capable LLMs, we can bridge this gap. We will treat images (or screenshots of document pages) as first-class citizens, retrieving them directly based on their visual and semantic content and passing them to a vision-capable LLM for reasoning.

To demonstrate this, we will build a pipeline that extracts insights from the charts and figures of the GitHub Octoverse 2025 survey, which simulates the type of information typically found in enterprise data.

The Jupyter Notebook for this tutorial is available on GitHub in our GenAI Showcase repository. To follow along, run the notebook in Google Colab (or similar), and refer to this tutorial for explanations of key code blocks.

Step 1: Install necessary libraries First, we need to set up our Python environment. We will install the voyageai client for generating embeddings and the anthropic client for our generative model.

....

It was a big year for the company—and more importantly, for the tens of thousands of customers and millions of developers who rely on MongoDB’s modern data platform for their most mission-critical workloads. At MongoDB, everything we do starts with our obsession with customers and their needs, and if there’s a theme to MongoDB’s 2025, it was (and will continue to be) enabling customer innovation and helping them succeed in the AI era.

So here are a few highlights of how MongoDB acted on behalf of customers in 2025. From the acquisition of Voyage AI to customer success across industries, a lot happened in 2025. Let’s go!*

*Read to the end for 2026 thoughts.

2025: The (MongoDB) year that was Voyage AI, modernization, and search In February, MongoDB announced the acquisition of Voyage AI, a pioneer in embedding and reranking models, to enhance the accuracy of AI applications. Integrating Voyage AI's advanced retrieval technology with MongoDB’s modern, AI-ready data platform addresses a critical challenge: LLM model hallucinations caused by a lack of context. By improving retrieval accuracy for specialized domains like finance and law, the integration enables businesses to deploy AI for mission-critical use cases.

To learn more, see the MongoDB Voyage AI page.

Then, in September, we launched MongoDB AMP, an AI-powered Application Modernization Platform. AMP is designed to accelerate the transformation of legacy applications through a combination of AI-powered tooling, a proven delivery framework, and expert guidance (tools, techniques, and talent) to help enterprises reduce technical debt and modernize 2-3 times faster.

Want more? Sure you do! Check out this short video.

MongoDB also announced the addition of search and vector search capabilities to MongoDB Community Edition and MongoDB Enterprise Server. This allows developers to build and test AI-native applications, including those using retrieval-augmented generation (RAG), in local or on-premises environments. Previously exclusive to MongoDB Atlas, these features enable secure, hybrid deployments where sensitive data can remain on-premises while still leveraging advanced search tools.

Here’s a (slightly less short) video about search and vector search on Enterprise Server.

Growing and scaling with MongoDB As noted, everything we do at MongoDB starts with our obsession with customers. 2025 was another banner year for customer success and innovation—we were inspired by what organizations of every shape and size, across industries and geographies, built with MongoDB in 2025. Here are just two of the many stories our customers shared in 2025; much more can be found in my colleague Katie Palmer’s blog series, Innovating with MongoDB.

Factory By combining the Atlas modern data platform with Voyage AI’s high-performance embeddings, the AI-native startup Factory—which uses AI agents called Droids to accelerate software development lifecycles for organizations—consolidated its fragmented tech stack. This enabled superior code retrieval, simplified operations, and provided the scalability needed to process billions of tokens daily.

McKesson McKesson, a global pharmaceutical distributor, replaced its monolithic legacy infrastructure with MongoDB Atlas to meet strict drug tracing mandates. By adopting our modern cloud data platform, McKesson scaled its operations 300x, managing tracking data for 1.2 billion containers annually without latency, and ensuring compliance and patient safety while reducing developer complexity.

For more, check out the video of McKesson at MongoDB.local NYC from September.

From niche NoSQL to enterprise powerhouse As senior MongoDB engineer and Technical Fellow Ashish Kumar put it earlier this year, “through a sustained and deliberate engineering effort,” MongoDB has gone from a (seemingly) niche NoSQL solution to a trusted enterprise standard, and now delivers “the high availability, tunable consistency, ACID transactions, and robust security that enterprises demand.”

A new era of leadership The face of MongoDB has also changed—our CFO, Mike Berry, joined the company in April, and Dev Ittycheria stepped down as CEO in November, after more than 11 years leading the company (including its 2017 IPO). In a LinkedIn post about his role, new MongoDB CEO CJ Desai noted that the company is “at the forefront of a new data revolution, unlocking the next wave of productivity and intelligence.”

“Having spent my career building and scaling technology platforms, I’ve always been drawn to companies defined by clarity of vision, relentless organic innovation, and a customer-first culture. MongoDB exemplifies all three,” said Desai.

We couldn’t agree more. Onward!

Reading the 2026 tea leaves So what might 2026 bring (for MongoDB and tech at large)? Here are a handful of our leaders’ predictions:

“As much as people want to talk about Artificial General Intelligence (AGI), we’re still in the phase where most AI use cases automate redundant tasks but benefit from human-in-the-loop checks. Organizations that use AI to complete work that historically is a drain on human resources—but then uses people to carefully verify what AI builds, apply governance frameworks, and maintain accountability across the data lifecycle—will be more successful.” —Pete Johnson, Field CTO, AI, MongoDB

“After years of inflated expectations and unsustainable spending, the AI industry is trapped in a bubble where companies reflexively attempt to deploy LLMs at every problem, driving up costs with minimal to no return. Businesses that break free from this spending cycle are the ones that understand the need to ground LLM responses in factual data and learn from prior mistakes. We believe the best way to do this will be with highly accurate embedding models and rerankers for reliable data retrieval.” —Frank Liu, Staff Product Manager, MongoDB

"In 2026, cloud independence will evolve from strategic preference to existential imperative across enterprises of every scale. The outages and disruptions of recent years have exposed a fundamental truth: in an always-on digital economy—where commerce, mobility, governance, and even public safety depend on uninterrupted access to cloud services—single-provider reliance is no longer a calculated risk, but a systemic vulnerability.

Compounding this is the inexorable rise of data sovereignty. Regulatory regimes worldwide now demand precise jurisdictional control over data residency, rendering rigid cloud commitments incompatible with compliance at global scale.

The defining competitive advantage will belong to organizations that transcend fragile prevention theater and engineer true infrastructural resilience: architectures inherently portable, data frictionlessly mobile, and operations autonomously sustained across heterogeneous clouds through AI-orchestrated redundancy.

In short, the winners will not merely mitigate downtime—they will design systems that render the concept obsolete." —Ben Cefalo, SVP, Head of Core Products, MongoDB

Happy holidays and happy New Year, everyone!

Queries are typically short, and their token-length distribution is highly skewed. As a result, query inference tends to be memory-bound rather than compute-bound. Query traffic is pretty spiky, so autoscaling is too slow. In sum, serving many short requests sequentially is highly inefficient.

In this blog post, we explore how batching can be used to serve queries more efficiently. We first discuss padding removal in modern inference engines, a key technique that enables effective batching. We then present practical strategies for forming batches and selecting an appropriate batch size. Finally, we walk through the implementation details and share the resulting performance improvements: a 50% reduction in GPU inference latency—despite using 3X fewer GPUs.

Padding removal makes effective batching possible Given the patterns of query traffic, one straightforward idea is: can we batch them to improve inference efficiency? Padding removal, supported in inference engines like vLLM and SGLang, makes efficient batching possible.

Most inference engines accept requests in the form (B, S), where B is the sequence number in the batch, and S is the maximum sequence length. Sequences should be padded to the max sequence length so that tensors line up. But that convenience comes at a cost: padding tokens do no useful work but still consume compute and memory bandwidth, so latency scales with B × S instead of the actual token count. With serving large volumes of short requests, this wastes a large share of compute and can inflate tail latency.Padding removal and variable-length processing fix this by concatenating all active sequences into one long "super sequence" of length T = Σtoken_count_i, where token_count_i is the token count in sequence i. Inference engines like vLLM and SGLang can process this combined sequence. Attention masks and position indices ensure that each sequence only attends to its own tokens. As a result, inference time now tracks T rather than B × S, aligning GPU work with what matters.

Proposal: token-count-based batching In Voyage AI, we proposed and built token-count-based batching, batching queries (short requests) by total token count in the batch (Σtoken_count_i), rather than by total request count or arbitrary time windows.

Time-window batching is inefficient when serving many short requests. Time-window batching swings between under- and over-filled batches depending on traffic. A short window keeps latency low but produces small, under-filled batches; a long window improves utilization but adds queueing delay. Traffic is bursty, so a single window size oscillates between under- and over-filling. Time-window batching introduces variability in resource utilization, causing the system to shift between memory-bound and compute-bound operations. Request-count batching has similar problems.

Diagram illustrating the difference between request-number-based batching and token-count-based batching. Figure 1. Request-number-based batching VS token-count-based batching. Token-count batching aligns the batch size (total token count in the batch) with the actual compute required. When many queries arrive close together, we group them by token counts so the GPU processes a larger combined workload in a single forward pass. Based on our experiment, token-count-based batching amortizes fixed costs, reduces per-request latency and cost, and increases throughput and model flops utilization (MFU).

What is the optimal batch size? Our inference-latency-vs-token-count profiling of query inference shows a clear pattern: latency is approximately flat up to a threshold (saturation point) and then becomes approximately linear. For small requests, fixed per-request overheads (like GPU scheduling, memory movement, pooling and normalization, etc.) dominate, and latency stays nearly constant; beyond that point, latency scales with token count. The threshold (saturation point) depends on factors like the model architecture, inference engines, and GPU. For our voyage-3 model running on A100, the threshold is about 600 tokens.

Line graph showing that GPU inference latency for the Voyage-3 model on A100 is approximately flat for batches up to about 600 tokens (the saturation point), and then increases linearly with the total token count. Figure 2. Inference latency vs token count for Voyage-3 on A100. Based on the data of inference latency vs token count, we can analyze FLOPs Utilization (MFU) vs. token count and throughput vs. token count, which are shown in the following graph. We observe that Model FLOPs Utilization (MFU) and throughput scale approximately linearly with token count until reaching a saturation point. Most of our queries inferences are in the memory-bound zone, far away from the saturation point.

Approximated diagram showing Model FLOPs Utilization (MFU) and throughput scaling linearly with token count up to a saturation point, and inference latency remaining flat until the saturation point, after which it also scales linearly. This illustrates how batching can shift inference from memory-bound to compute-bound. Figure 3. Approximated diagram of MFU/throughput/inference latency vs token count. Batching short requests can move the inference from memory-bound to compute-bound. If we choose the saturation point in Figure 3 as the batch size (total token count in the batch), the latency and throughput/MFU can be balanced and optimized.

Queue design: enabling token-count-based batching Token-count–based batching needs a data system that does more than simple FIFO delivery. The system has to attach an estimated token_count to each request, peek across pending requests, and then atomically claim a subset whose total tokens fit the optimal batch size (Σtoken_count_i ≤ optimal_batch_size). Without these primitives, we either underfill the GPU—wasting fixed overheads—or overfill it and spike tail latency.

General-purpose brokers like RabbitMQ and Kafka are excellent at durability, fan-out, and delivery, but their batching knobs are message count/bytes, not tokens. RabbitMQ’s prefetch is request-count-based, and messages are pushed to consumers, so there’s no efficient way to peek and batch requests by Σtoken_count_i. Kafka batches by bytes/messages within a partition; token count varies with text and tokenizer, so there is no efficient way to batch requests by Σtoken_count_i.

So there are two practical paths to make token-count-based batching work. One is to place a lightweight aggregator in front of Kafka/RabbitMQ that consumes batches by token counts and then dispatches batches to model servers. The other is to use a store that naturally supports fast peek + conditional batching—for example, Redis with Lua script. In our implementation, we use Redis because it lets us atomically “pop up to the optimal batch size” and set per-item TTLs within a single lua script call. Whatever we choose, the essential requirement is the same: the queue must let our system see multiple pending items, batch by Σtoken_count_i, and claim them atomically to keep utilization stable and latency predictable.

Our system enqueues each embedding query request into a Redis list as:

Code Snippet 1 <token_count>::::

Model servers call lua script atomically to fetch a batch of requests until the optimal batch size is reached. The probability of Redis losing data is very low. In the rare case that it does happen, users may receive 503 Service Unavailable errors and can simply retry. When QPS is low, batches are only partially filled and GPU utilization remains low, but latency still improves.

Diagram illustrating the token-count-based batching implementation, where incoming embedding query requests are enqueued into a Redis list and model servers atomically fetch a batch of requests up to the optimal total token count using a Lua script. Figure 4. Batching implementation. Results We ran a production experiment on the Voyage-3-Large model serving, comparing our new pipeline (query batching + vLLM) against our old pipeline (no batching + Hugging Face Inference). We saw a 50% reduction in GPU inference latency—despite using 3X fewer GPUs.

We gradually onboarded 7+ models to the above query batching solution, and saw the following results (note that these results are based on our specific implementations of the “new” and “old” pipelines, and are not necessarily generalizable):

vLLM reduces GPU inference time by up to ~20 ms for most of our models.

GPU utilization and MFU increase, reflecting reduced padding, better amortization of per-batch overhead, and inference moving closer to the compute-bound regime.

Throughput improves by up to 8× via token-count–based batching.

Some model servers see P90 end-to-end latency drop by 60+ ms as queuing time is reduced under resource contention.

P90 end-to-end latency is more stable during traffic spikes, even with fewer GPUs.

In summary, combining padding removals with token-count-based batching improves throughput and reduces latency, while improving resource utilization and lowering operational costs during short query embedding inference.

This is the hardest email I have ever had to write to all of you. If you have not seen the announcement, I have decided to retire as CEO. Effective November 10, 2025, Chirantan “CJ” Desai will become the new CEO of MongoDB.

This was not an easy decision for me. The process to get to this point has been deeply emotional, as I care profoundly about MongoDB and the people who have made the company what it is today.

This news may come as a surprise, and for some, perhaps even a shock. That’s natural. Leadership transitions can evoke a range of reactions. I want to share why this is happening, and why it’s the right thing for MongoDB.

Every personnel change, including the most senior leadership changes, involves two key decisions: first, recognizing that it is the right time for change, and second, selecting the best person to replace the person leaving. This email is intended to explain both decisions.

Earlier this year, as part of our regular succession planning process, the Board and I discussed my long-term commitment. They asked if I would continue as CEO for another five years. After many conversations with my family and the Board, I realized I could not make that commitment. Some CEOs see their title as their identity. I do not. My core responsibility is to serve in the company's best interests. The company is primed for a new leader. One with a fresh perspective, grounded in experience and skills needed to guide MongoDB through its next evolution as a company, what we call MongoDB 3.0.

Consequently, I informed the Board that I would commit to two more years to help find a successor. That began the search process for a suitable successor. To our surprise and delight, what we thought would easily take 12 to 24 months happened much faster than anyone expected. After engaging with multiple qualified candidates, we found the right successor in CJ.

CJ is uniquely qualified for this role. CJ brings the rare growth-at-scale experience that will help continue to build MongoDB into an iconic technology company. At ServiceNow, he was the only executive to work directly with three of its highly regarded public company CEOs and played a pivotal role in organically scaling the company from just over $1 billion to more than $10 billion in revenue. Only a handful of independent software companies have ever reached that milestone. CJ helped transform ServiceNow from a product company to a platform company, scaled engineering, drove go-to-market excellence, and engaged deeply with investors. More recently, as President of Product and Engineering at Cloudflare, he helped fuel strong growth and stock performance.

CJ also possesses the personal qualities needed to succeed as CEO. He is humble, eager to learn, and wants to draw on the perspectives of the people at MongoDB and other stakeholders to inform his thinking. This blend of experience, judgment, and character gives me full confidence that he is well-equipped to lead MongoDB through its next phase of growth.

I often think of MongoDB’s journey as a long and extraordinary expedition. For the past eleven years, I have had the privilege of serving as its guide, helping chart the course, rally the team, and climb together through both calm and challenging terrain. Along the way, we have reached remarkable summits and proven what is possible through relentless innovation, persistence, and teamwork. Now it is time for a new guide to lead the next stage of the ascent and take MongoDB to even greater heights. CJ is the right leader to take MongoDB to the next summit.

MongoDB is on a strong footing, with a clear strategy, an exceptional leadership team, a product platform that is more relevant than ever, and a business that is executing well. The rise of AI and the explosion of data-intensive applications play directly to MongoDB’s strengths. Our technology sits at the center of how modern applications are built and how organizations will harness data to power intelligent, adaptive systems. I am confident MongoDB is perfectly positioned to capture this next wave of innovation.

As for me, I am not running away from MongoDB or leaving to join another company as CEO. I will remain on the Board and work closely with CJ to ensure a seamless transition. Over the years, this role has demanded an enormous amount of focus and energy; as a result, there are many things I’ve missed doing along the way. I’m looking forward to being more present for those moments — from simple time with my family to experiences and travel we’ve long put off. I plan to hold on to my MongoDB stock, as I firmly believe in the people and the opportunity, knowing that MongoDB’s best days are ahead of it.

Yes, change can be unsettling. I’m sure you will have many questions about this change, such as why now, why CJ is the best person to lead the company, and what this means for you. We will hold an all-hands meeting tomorrow at 10:30AM ET to discuss this transition, introduce CJ and take your questions.

That being said, I want to emphasize that the right change at the right time is how great companies get stronger. Just as a championship team refreshes its roster to stay competitive, MongoDB is bringing in new leadership, including other recent C-suite leaders who came before CJ, to drive our next phase of growth. This is not an ending; it’s the founding of a new moment.

I am incredibly proud of what we have built together and genuinely excited about what lies ahead with CJ leading us forward. I also want to thank each of you for making this journey so meaningful. Words cannot fully capture my gratitude for your passion, creativity, and belief in building something truly special.

I have often said that I want MongoDB to be an inflection point in people’s careers, a place where they can grow, take risks, and do the best work of their lives. I can say without hesitation that it has been exactly that for me. The skills I have developed, the experiences I have gained, and the relationships I have formed here have shaped me more than any other chapter in my professional life. I will carry them with me always, and will continue to cheer for and support MongoDB every step of the way.

--Dev

Speaking at MongoDB .local Bengaluru in July 2025, Pradeep Sharma, Head of Technology at Cars24, shared how MongoDB has been a key driver of Car24’s digital transformation journey. Specifically, he highlighted two recent use cases that show how MongoDB Atlas has helped Cars24 scale, improve its search capabilities, and reduce its architectural complexity.

Matching the growing scale with simplified and expanded search

Cars24 has operations in multiple countries, and a diverse customer base. Over the years, the company has used customer data, behavior analytics, and operational workflows to build, evolving from being a platform for buying and selling cars, to an end-to-end ecosystem, supported by a hub of interconnected systems.

At the start of its journey, Cars24 relied on legacy databases for managing and searching data, such as Postgres. Their relational database set-up would store information, synchronize the data to a separate “bolt-on” search engine (such as Elasticsearch), manually indexing it, and then querying the index.

While initially effective for a small application ecosystem, these processes became bottlenecked as the organization’s services grew. Multiple engineering teams piped data into a single search index, which often resulted in synchronization challenges and overwhelming administrative overhead.

Cars24 faced three core limitations with this setup:

• Lower developer productivity: Exponential effort was spent maintaining pipelines and synchronizing procedures. Developers had little bandwidth for building business features or innovation.

• Architectural complexity: Ensuring data sync consistency required multiple pipelines and race logic. This led to inefficiencies in real-time dashboard updates for agents.

• Operational overhead: Maintaining separate systems for database and search—alongside provisioning, patching, scaling, and monitoring—strained resources.

Seeking an integrated approach, Cars24 embraced MongoDB Atlas, hosted on Google Cloud. MongoDB Atlas would serve as a single, consistent, modern database and embedded search solution, powered by Apache Lucene.

MongoDB Atlas Search also enabled Cars24 to run queries directly in the database. This eliminated the need to synchronise data between systems while delivering real-time results.

This unified approach allowed the company’s developers to transition from managing complex synchronization mechanisms to building applications. Furthermore, the reduced administrative overhead enabled Cars24 to consolidate the team’s efforts, and to streamline query execution across the ecosystem.

Thanks to MongoDB Atlas and MongoDB Atlas Search, Cars24 was able to:

• Avoid "synchronization tax”: Switching to MongoDB Atlas eliminated the need for data synchronization and the additional tooling this mandated. Real-time searches can be performed from a single interface and workflow.

• Deliver new search features faster: By using a single, unified API across database and search operations, new features can be delivered rapidly.

• Work with a fully managed platform: With MongoDB Atlas, Cars24’s engineers can focus more on application development and building products, rather than thinking about managing indexes, syncing, and more.

Following this successful migration, Cars24 decided to also use MongoDB Atlas to replace one of its legacy databases, ArangoDB. The switch to MongoDB Atlas eliminated major roadblocks for other critical search capabilities.

From ArangoDB to MongoDB: Streamlined operations and 50% cost savings

As Cars24 scaled new services globally, it encountered limitations with its geospatial search solution, which was based on ArangoDB. This included performance bottlenecks, weak transactions as it was difficult to guarantee consistent data operations, and a limited ecosystem which meant that scaling developer onboarding and troubleshooting became increasingly onerous.
Moving to MongoDB Atlas enabled Cars24 to transition its geospatial services, consolidating its data storage and search capabilities under a single, versatile platform.

“We now have a highly available architecture, and an amazing team at MongoDB that has our back,” said Sharma.

MongoDB offered a proven architecture for high availability, scalability, and real-world production readiness:

• Enhanced scalability: MongoDB’s ability to scale massive workloads supports Cars24’s growing global presence.

• Reliable transactions: MongoDB provides robust multi-document ACID transactions across shards, meeting mission-critical needs.

• Streamlined operations: MongoDB offers a single platform that is not limited to a database only. By consolidating its geospatial search workload under MongoDB, Cars24 has reduced maintenance and operational overhead.

Not only did Cars24 cut costs in half by moving to MongoDB, but the widespread market adoption of MongoDB Atlas also means that Cars24 can continue to rapidly onboard developers familiar with MongoDB, a recruiting priority for Cars24’s growing development team.

“To give you an idea, one of our business units had a developer team of less than 10 about a year ago. Now they are a triple-digit team,” said Sharma. “If we are going to keep introducing new developers, for a product coming up or scaling up, it becomes very important to focus on the community skills and support provided by our technology partner.”

“Now that we have moved from ArangoDB to MongoDB Atlas, our developers are the happiest,” he added.

Cars24 is now looking to consolidate even more of its application and data workflows under MongoDB Atlas. With the growing number of developers joining Cars24’s engineering teams, plans are to utilize MongoDB Atlas further to enhance productivity, scalability, and data-driven insights.

In Part 1, we improved application performance by concatenating fields, changing data types, and shortening field names. In Part 2, we implemented the Bucket Pattern and Computed Pattern and optimized the aggregation pipeline to achieve even better performance.

In this final article, we address the issues and improvements identified in appV5R4. Specifically, we focus on reducing the document size in our application to alleviate the disk throughput bottleneck on the MongoDB server. This reduction will be accomplished by adopting a dynamic schema and modifying the storage compression algorithm.

All the application versions and revisions from this article were developed by a senior MongoDB developer, as they are built on all the previous versions and utilize the Dynamic Schema pattern, which isn't very common to see.

Application version 6 revision 0 (appV6R0): A dynamic monthly bucket document

As mentioned in the Issues and Improvements of appV5R4 from the previous article, the primary limitation of our MongoDB server is its disk throughput. To address this, we need to reduce the size of the documents being stored.

Consider the following document from appV5R3, which has provided the best performance so far:

const document = {
  _id: Buffer.from("...01202202"),
  items: [
    { date: new Date("2022-06-05"), a: 10, n: 3 },
    { date: new Date("2022-06-16"), p: 1, r: 1 },
    { date: new Date("2022-06-27"), a: 5, r: 1 },
    { date: new Date("2022-06-29"), p: 1 },
  ],
};

The items array in this document contains only four elements, but on average, it will have around 10 elements, and in the worst-case scenario, it could have up to 90 elements. These elements are the primary contributors to the document size, so they should be the focus of our optimization efforts.

One commonality among the elements is the presence of the date field, with its value including the year and month, for the previous document. By rethinking how this field and its value could be stored, we can reduce storage requirements.

An unconventional solution we could use is:

• Changing the items field type from an array to a document.

• Using the date value as the field name in the items document.

• Storing the status totals as the value for each date field.

Here is the previous document represented using the new schema idea:

const document = {
  _id: Buffer.from("...01202202"),
  items: {
    20220605: { a: 10, n: 3 },
    20220616: { p: 1, r: 1 },
    20220627: { a: 5, r: 1 },
    20220629: { p: 1 },
  },
};

While this schema may not significantly reduce the document size compared to appV5R3, we can further optimize it by leveraging the fact that the year is already embedded in the _id field. This eliminates the need to repeat the year in the field names of the items document.

With this approach, the items document adopts a Dynamic Schema, where field names encode information and are not predefined.

To demonstrate various implementation possibilities, we will revisit all the bucketing criteria used in the appV5RX implementations, starting with appV5R0.

For appV6R0, which builds upon appV5R0 but uses a dynamic schema, data is bucketed by year and month. The field names in the items document represent only the day of the date, as the year and month are already stored in the _id field.

A detailed explanation of the bucketing logic and functions used to implement the current application can be found in the appV5R0 introduction.

The following document stores data for January 2022 (2022-01-XX), applying the newly presented idea:

const document = {
  _id: Buffer.from("...01202201"),
  items: {
    "05": { a: 10, n: 3 },
    16: { p: 1, r: 1 },
    27: { a: 5, r: 1 },
    29: { p: 1 },
  },
};

Schema

The application implementation presented above would have the following TypeScript document schema denominated SchemaV6R0:

export type SchemaV6R0 = {
  _id: Buffer;
  items: Record<
    string,
    {
      a?: number;
      n?: number;
      p?: number;
      r?: number;
    }
  >;
};

Bulk upsert

Based on the specification presented, we have the following updateOne operation for each event generated by this application version:

const DD = getDD(event.date); // Extract the `day` from the `event.date`

const operation = {
  updateOne: {
    filter: { _id: buildId(event.key, event.date) }, // key + year + month
    update: {
      $inc: {
        [`items.${DD}.a`]: event.approved,
        [`items.${DD}.n`]: event.noFunds,
        [`items.${DD}.p`]: event.pending,
        [`items.${DD}.r`]: event.rejected,
      },
    },
    upsert: true,
  },
};

filter:

• Target the document where the _id field matches the concatenated value of key, year, and month.

• The buildId function converts the key+year+month into a binary format.

update:

• Uses the $inc operator to increment the fields corresponding to the same DD as the event by the status values provided.

• If a field does not exist in the items document and the event provides a value for it, $inc treats the non-existent field as having a value of 0 and performs the operation.

• If a field exists in the items document but the event does not provide a value for it (i.e., undefined), $inc treats it as 0 and performs the operation.

upsert:

• Ensures a new document is created if no matching document exists.

Get reports

To fulfill the Get Reports operation, five aggregation pipelines are required, one for each date interval. Each pipeline follows the same structure, differing only in the filtering criteria in the $match stage:

const pipeline = [
  { $match: docsFromKeyBetweenDate },
  { $addFields: buildTotalsField },
  { $group: groupSumTotals },
  { $project: { _id: 0 } },
];

The complete code for this aggregation pipeline is quite complicated. Because of that, we will have just a pseudocode for it here.

1: { $match: docsFromKeyBetweenDate }

• Range-filters documents by _id to retrieve only buckets within the report date range. It has the same logic as appV5R0.

2: { $addFields: buildTotalsField }

• The logic is similar to the one used in the Get Reports of appV5R3.

• The $objectToArray operator is used to convert the items document into an array, enabling a $reduce operation.

• Filtering the items fields within the report's range involves extracting the year and month from the _id field and the day from the field names in the items document.

• The following JavaScript code is logic equivalent to the real aggregation pipeline code.

// Equivalent JavaScript logic:
const [MM] = _id.slice(-2).toString(); // Get month from _id
const [YYYY] = _id.slice(-6, -2).toString(); // Get year from _id
const items_array = Object.entries(items); // Convert the object to an array of [key, value]

const totals = items_array.reduce(
  (accumulator, [DD, status]) => {
    let statusDate = new Date(`${YYYY}-${MM}-${DD}`);

    if (statusDate >= reportStartDate && statusDate < reportEndDate) {
      accumulator.a += status.a || 0;
      accumulator.n += status.n || 0;
      accumulator.p += status.p || 0;
      accumulator.r += status.r || 0;
    }

    return accumulator;
  },
  { a: 0, n: 0, p: 0, r: 0 }
);

3: { $group: groupCountTotals }

• Group the totals of each document in the pipeline into final status totals using $sum operations.

4: { $project: { _id: 0 } }

• Format the resulting document to have the reports format.

Indexes

No additional indexes are required, maintaining the single _id index approach established in the appV4 implementation.

Initial scenario statistics

Collection statistics

To evaluate the performance of appV6R0, we inserted 500 million event documents into the collection using the schema and Bulk Upsert function described earlier. For comparison, the tables below also include statistics from previous comparable application versions:

Event statistics

To evaluate the storage efficiency per event, the Event Statistics are calculated by dividing the total data size and index size by the 500 million events.

It is challenging to make a direct comparison between appV6R0 and appV5R0 from a storage perspective. The appV5R0 implementation is the simplest bucketing possible, where event documents were merely appended to the items array without bucketing by day, as is done in appV6R0.

However, we can attempt a comparison between appV6R0 and appV5R3, the best solution so far. In appV6R0, data is bucketed by month, whereas in appV5R3, it is bucketed by quarter. Assuming document size scales linearly with the bucketing criteria (though this is not entirely accurate), the appV6R0 document would be approximately 3 * 125 = 375 bytes, which is 9.4% smaller than appV5R3.

Another indicator of improvement is the Data Size/Events metric in the Event Statistics table. For appV6R0, each event uses an average of 23.8 bytes, compared to 27.7 bytes for appV5R3, representing a 14.1% reduction in size.

Load test results

Executing the load test for appV6R0 and plotting it alongside the results for appV5R0 and Desired rates, we have the following results for Get Reports and Bulk Upsert.

Get Reports rates

The two versions exhibit very similar rate performance, with appV6R0 showing slight superiority in the second and third quarters, while appV5R0 is superior in the first and fourth quarters.

Get Reports latency

The two versions exhibit very similar latency performance, with appV6R0 showing slight advantages in the second and third quarters, while appV5R0 is superior in the first and fourth quarters.

Bulk Upsert rates

Both versions have similar rate values, but it can be seen that appV6R0 has a small edge compared to appV5R0.

Bulk Upsert latency

Although both versions have similar latency values for the first quarter of the test, for the final three-quarters, appV6R0 has a clear advantage over appV5R0.

Performance summary

Despite the significant reduction in document and storage size achieved by appV6R0, the performance improvement was not as substantial as expected. This suggests that the bottleneck in the application when bucketing data by month may not be related to disk throughput.

Examining the collection stats table reveals that the index size for both versions is close to 3GB. This is near the 4GB of available memory on the machine running the database and exceeds the 1.5GB allocated by WiredTiger for cache. Therefore, it is likely that the limiting factor in this case is memory/cache rather than document size, which explains the lack of a significant performance improvement.

Issues and improvements

To address the limitations observed in appV6R0, we propose adopting the same line of improvements applied from appV5R0 to appV5R1. Specifically, we will bucket the events by quarter in appV6R1. This approach not only follows the established pattern of enhancements but also aligns with the need to optimize performance further.

As highlighted in the Load Test Results, the current bottleneck lies in the size of the index relative to the available cache/memory. By increasing the bucketing interval from month to quarter, we can reduce the number of documents by approximately a factor of three. This reduction will, in turn, decrease the number of index entries by the same factor, leading to a smaller index size.

Application version 6 revision 1 (appV6R1): A dynamic quarter bucket document

As discussed in the previous Issues and Improvements section, the primary bottleneck in appV6R0 was the index size nearing the memory capacity of the machine running MongoDB. To mitigate this issue, we propose increasing the bucketing interval from a month to a quarter for appV6R1, following the approach used in appV5R1.

This adjustment aims to reduce the number of documents and index entries by approximately a factor of three, thereby decreasing the overall index size. By adopting a quarter-based bucketing strategy, we align with the established pattern of enhancements applied in appV5R1 versions while addressing the specific memory/cache constraints identified in appV6R0.

The implementation of appV6R1 retains most of the code from appV6R0, with the following key differences:

• The _id field will now be composed of key+year+quarter.

• The field names in the items document will encode both month and day, as this information is necessary for filtering date ranges in the Get Reports operation.

The following example demonstrates how data for June 2022 (2022-06-XX), within the second quarter (Q2), is stored using the new schema:

const document = {
  _id: Buffer.from("...01202202"),
  items: {
    "0605": { a: 10, n: 3 },
    "0616": { p: 1, r: 1 },
    "0627": { a: 5, r: 1 },
    "0629": { p: 1 },
  },
};

Schema

The application implementation presented above would have the following TypeScript document schema denominated SchemaV6R0:

export type SchemaV6R0 = {
  _id: Buffer;
  items: Record<
    string,
    {
      a?: number;
      n?: number;
      p?: number;
      r?: number;
    }
  >;
};

Bulk upsert

Based on the specification presented, we have the following updateOne operation for each event generated by this application version:

const MMDD = getMMDD(event.date); // Extract the month (MM) and day(DD) from the `event.date`

const operation = {
  updateOne: {
    filter: { _id: buildId(event.key, event.date) }, // key + year + quarter
    update: {
      $inc: {
        [`items.${MMDD}.a`]: event.approved,
        [`items.${MMDD}.n`]: event.noFunds,
        [`items.${MMDD}.p`]: event.pending,
        [`items.${MMDD}.r`]: event.rejected,
      },
    },
    upsert: true,
  },
};

This updateOne operation has a similar logic to the one in appV6R0, with the only differences being the filter and update criteria.

filter:

• Target the document where the _id field matches the concatenated value of key, year, and quarter.

• The buildId function converts the key+year+quarter into a binary format.

update:

• Uses the $inc operator to increment the fields corresponding to the same MMDD as the event by the status values provided.

Get reports

To fulfill the Get Reports operation, five aggregation pipelines are required, one for each date interval. Each pipeline follows the same structure, differing only in the filtering criteria in the $match stage:

const pipeline = [
  { $match: docsFromKeyBetweenDate },
  { $addFields: buildTotalsField },
  { $group: groupSumTotals },
  { $project: { _id: 0 } },
];

This aggregation operation has a similar logic to the one in appV6R0, with the only differences being the implementation in the $addFields stage.

{ $addFields: itemsReduceAccumulator }:

• A similar implementation to the one in appV6R0

• The difference relies on extracting the value of year (YYYY) from the _id field and the month and day (MMDD) from the field name.

• The following JavaScript code is logic equivalent to the real aggregation pipeline code.

const [YYYY] = _id.slice(-6, -2).toString(); // Get year from _id
const items_array = Object.entries(items); // Convert the object to an array of [key, value]

const totals = items_array.reduce(
  (accumulator, [MMDD, status]) => {
    let [MM, DD] = [MMDD.slice(0, 2), MMDD.slice(2, 4)];
    let statusDate = new Date(`${YYYY}-${MM}-${DD}`);

    if (statusDate >= reportStartDate && statusDate < reportEndDate) {
      accumulator.a += status.a || 0;
      accumulator.n += status.n || 0;
      accumulator.p += status.p || 0;
      accumulator.r += status.r || 0;
    }

    return accumulator;
  },
  { a: 0, n: 0, p: 0, r: 0 }
);

Indexes

No additional indexes are required, maintaining the single _id index approach established in the appV4 implementation.

Initial scenario statistics

Collection statistics

To evaluate the performance of appV6R1, we inserted 500 million event documents into the collection using the schema and Bulk Upsert function described earlier. For comparison, the tables below also include statistics from previous comparable application versions:

Event statistics

To evaluate the storage efficiency per event, the Event Statistics are calculated by dividing the total data size and index size by the 500 million events.

In the previous Initial Scenario Statistics analysis, we assumed that document size would scale linearly with the bucketing range. However, this assumption proved inaccurate. The average document size in appV6R1 is approximately twice as large as in appV6R0, even though it stores three times more data. Already a win for this new implementation.

Since appV6R1 buckets data by quarter at the document level and by day within the items sub-document, a fair comparison would be with appV5R3, the best-performing version so far. From the tables above, we observe a significant improvement in Document Size and consequently Data Size when transitioning from appV5R3 to appV6R1. Specifically, there was a 31.4% reduction in Document Size. From an index size perspective, there was no change, as both versions bucket events by quarter.

Load test results

Executing the load test for appV6R0 and plotting it alongside the results for appV5R0 and Desired rates, we have the following results for Get Reports and Bulk Upsert.

Get Reports rates

For the first three-quarters of the test, both versions have similar rate values, but, for the final quarter, appV6R1 has a notable edge over appV5R3.

Get Reports latency

The two versions exhibit very similar latency performance, with appV6R0 showing slight advantages in the second and third quarters, while appV5R0 is superior in the first and fourth quarters.

Bulk Upsert rates

Both versions have similar rate values, but it can be seen that appV6R0 has a small edge compared to appV5R0.

Bulk Upsert latency

Although both versions have similar latency values for the first quarter of the test, for the final three-quarters, appV6R0 has a clear advantage over appV5R0.

Performance summary

Despite the significant reduction in document and storage size achieved by appV6R0, the performance improvement was not as substantial as expected. This suggests that the bottleneck in the application when bucketing data by month may not be related to disk throughput.

Examining the collection stats table reveals that the index size for both versions is close to 3GB. This is near the 4GB of available memory on the machine running the database and exceeds the 1.5GB allocated by WiredTiger for cache. Therefore, it is likely that the limiting factor in this case is memory/cache rather than document size, which explains the lack of a significant performance improvement.

Issues and improvements

To address the limitations observed in appV6R0, we propose adopting the same line of improvements applied from appV5R0 to appV5R1. Specifically, we will bucket the events by quarter in appV6R1. This approach not only follows the established pattern of enhancements but also aligns with the need to optimize performance further.

As highlighted in the Load Test Results, the current bottleneck lies in the size of the index relative to the available cache/memory. By increasing the bucketing interval from month to quarter, we can reduce the number of documents by approximately a factor of three. This reduction will, in turn, decrease the number of index entries by the same factor, leading to a smaller index size.

Application version 6 revision 1 (appV6R1): A dynamic quarter bucket document

As discussed in the previous Issues and Improvements section, the primary bottleneck in appV6R0 was the index size nearing the memory capacity of the machine running MongoDB. To mitigate this issue, we propose increasing the bucketing interval from a month to a quarter for appV6R1, following the approach used in appV5R1.

This adjustment aims to reduce the number of documents and index entries by approximately a factor of three, thereby decreasing the overall index size. By adopting a quarter-based bucketing strategy, we align with the established pattern of enhancements applied in appV5R1 versions while addressing the specific memory/cache constraints identified in appV6R0.

The implementation of appV6R1 retains most of the code from appV6R0, with the following key differences:

• The _id field will now be composed of key+year+quarter.

• The field names in the items document will encode both month and day, as this information is necessary for filtering date ranges in the Get Reports operation.

The following example demonstrates how data for June 2022 (2022-06-XX), within the second quarter (Q2), is stored using the new schema:

const document = {
  _id: Buffer.from("...01202202"),
  items: {
    "0605": { a: 10, n: 3 },
    "0616": { p: 1, r: 1 },
    "0627": { a: 5, r: 1 },
    "0629": { p: 1 },
  },
};

Schema

The application implementation presented above would have the following TypeScript document schema denominated SchemaV6R0:

export type SchemaV6R0 = {
  _id: Buffer;
  items: Record<
    string,
    {
      a?: number;
      n?: number;
      p?: number;
      r?: number;
    }
  >;
};

Bulk upsert

Based on the specification presented, we have the following updateOne operation for each event generated by this application version:

const MMDD = getMMDD(event.date); // Extract the month (MM) and day(DD) from the `event.date`

const operation = {
  updateOne: {
    filter: { _id: buildId(event.key, event.date) }, // key + year + quarter
    update: {
      $inc: {
        [`items.${MMDD}.a`]: event.approved,
        [`items.${MMDD}.n`]: event.noFunds,
        [`items.${MMDD}.p`]: event.pending,
        [`items.${MMDD}.r`]: event.rejected,
      },
    },
    upsert: true,
  },
};

This updateOne operation has a similar logic to the one in appV6R0, with the only differences being the filter and update criteria.

filter:

• Target the document where the _id field matches the concatenated value of key, year, and quarter.

• The buildId function converts the key+year+quarter into a binary format.

update:

• Uses the $inc operator to increment the fields corresponding to the same MMDD as the event by the status values provided.

Get reports

To fulfill the Get Reports operation, five aggregation pipelines are required, one for each date interval. Each pipeline follows the same structure, differing only in the filtering criteria in the $match stage:

const pipeline = [
  { $match: docsFromKeyBetweenDate },
  { $addFields: buildTotalsField },
  { $group: groupSumTotals },
  { $project: { _id: 0 } },
];

This aggregation operation has a similar logic to the one in appV6R0, with the only differences being the implementation in the $addFields stage.

{ $addFields: itemsReduceAccumulator }:

• A similar implementation to the one in appV6R0

• The difference relies on extracting the value of year (YYYY) from the _id field and the month and day (MMDD) from the field name.

• The following JavaScript code is logic equivalent to the real aggregation pipeline code.

const [YYYY] = _id.slice(-6, -2).toString(); // Get year from _id
const items_array = Object.entries(items); // Convert the object to an array of [key, value]

const totals = items_array.reduce(
  (accumulator, [MMDD, status]) => {
    let [MM, DD] = [MMDD.slice(0, 2), MMDD.slice(2, 4)];
    let statusDate = new Date(`${YYYY}-${MM}-${DD}`);

    if (statusDate >= reportStartDate && statusDate < reportEndDate) {
      accumulator.a += status.a || 0;
      accumulator.n += status.n || 0;
      accumulator.p += status.p || 0;
      accumulator.r += status.r || 0;
    }

    return accumulator;
  },
  { a: 0, n: 0, p: 0, r: 0 }
);

Indexes

No additional indexes are required, maintaining the single _id index approach established in the appV4 implementation.

Initial scenario statistics

Collection statistics

To evaluate the performance of appV6R1, we inserted 500 million event documents into the collection using the schema and Bulk Upsert function described earlier. For comparison, the tables below also include statistics from previous comparable application versions:

Event statistics

To evaluate the storage efficiency per event, the Event Statistics are calculated by dividing the total data size and index size by the 500 million events.

As expected, we had an 11.2% increase in the Document Size by adding a totals field in each document of appV6R2. When comparing to appV5R3, we still have a reduction of 23.9% in the Document Size. Let's review the Load Test Results to see if the trade-off between storage and computation cost is worthwhile.

Load test results

Executing the load test for appV6R2 and plotting it alongside the results for appV6R1 and Desired rates, we have the following results for Get Reports and Bulk Upsert.

Get Reports rates

We can see that appV6R2 has better rates than appV6R1 throughout the test, but it’s still not reaching the top rate of 250 reports per second.

Get Reports latency

As shown in the rates graph, appV6R2 consistently provides lower latency than appV6R1 throughout the test.

Bulk Upsert rates

Both versions exhibit very similar rate values throughout the test, with appV6R2 performing slightly better than appV6R1 in the final 20 minutes, yet still failing to reach the desired rate.

Bulk Upsert latency

Although appV6R2 had better rate values than appV6R1, their latency performance is not conclusive, with appV6R2 being superior in the first and final quarters and appV6R1 in the second and third quarters.

Performance summary

The two "maybes" from the previous Issues and Improvements made up for their promises, and we got the best performance for appV6R2 when comparing to appV6R1. This is the redemption of the Computed Pattern applied on a document level. This revision is one of my favorites because it shows that the same optimization on very similar applications can lead to different results. In our case, the difference was caused by the application being very bottlenecked by the disk throughput.

Issues and improvements

Let's tackle the last improvement on an application level. Those paying close attention to the application versions may have already questioned it. In every Get Reports section, we have "To fulfill the Get Reports operation, five aggregation pipelines are required, one for each date interval." Do we really need to run five aggregation pipelines to generate the reports document? Isn't there a way to calculate everything in just one operation? The answer is yes, there is.

The reports documents are composed of fields oneYear, threeYears, fiveYears, sevenYears, and tenYears, where each one was generated by its respective aggregation pipeline until now. Generating the reports this way is a waste of processing power because we are doing some part of the calculation multiple times. For example, to calculate the status totals for tenYears, we will also have to calculate the status totals for the other fields, as from a date range perspective, they are all contained in the tenYears date range.

So, for our next application revision, we'll condense the Get Reports five aggregation pipelines into one, avoiding wasting processing power on repeated calculation.

Application version 6 revision 3 (appV6R3): Getting everything at once

As discussed in the previous Issues and Improvements section, in this revision, we'll improve the performance of our application by changing the Get Reports functionality to generate the reports document using only one aggregation pipeline instead of five.

The rationale behind this improvement is that when we generate the tenYears totals, we have also calculated the other totals, oneYear, threeYears, fiveYears, and sevenYears. As an example, when we request to Get Reports with the key ...0001 with the date 2022-01-01, the totals will be calculated with the following date range:

• oneYear: from 2021-01-01 to 2022-01-01

• threeYears: from 2020-01-01 to 2022-01-01

• fiveYears: from 2018-01-01 to 2022-01-01

• sevenYears: from 2016-01-01 to 2022-01-01

• tenYear: from 2013-01-01 to 2022-01-01

As we can see from the list above, the date range for tenYears encompasses all the other date ranges.

Although we successfully implemented the Computed Pattern in the previous revision, appV6R2, achieving better results than appV6R1, we will not use it as a base for this revision. There were two reasons for that:

1. Based on the results of our previous implementation of the Computed Pattern on a document level, from appV5R3 to appV5R4, I didn't expect it to get better results.

2. Implementing Get Reports to retrieve the reports document through a single aggregation pipeline, utilizing pre-computed field totals generated by the Computed Pattern would require significant effort. By the time of the latest versions of this series, I just wanted to finish it.

So, this revision will be built based on the appV6R1.

Schema

The application implementation presented above would have the following TypeScript document schema denominated SchemaV6R0:

export type SchemaV6R0 = {
  _id: Buffer;
  items: Record<
    string,
    {
      a?: number;
      n?: number;
      p?: number;
      r?: number;
    }
  >;
};

Bulk upsert

Based on the specifications, the following bulk updateOne operation is used for each event generated by the application:

const YYYYMMDD = getYYYYMMDD(event.date); // Extract the year(YYYY), month(MM), and day(DD) from the `event.date`

const operation = {
  updateOne: {
    filter: { _id: buildId(event.key, event.date) }, // key + year + quarter
    update: {
      $inc: {
        [`items.${YYYYMMDD}.a`]: event.approved,
        [`items.${YYYYMMDD}.n`]: event.noFunds,
        [`items.${YYYYMMDD}.p`]: event.pending,
        [`items.${YYYYMMDD}.r`]: event.rejected,
      },
    },
    upsert: true,
  },
};

This updateOne has almost exactly the same logic as the one for appV6R1. The difference is that the name of the fields in the items document will be created based on year, month, and day (YYYYMMDD) instead of just month and day (MMDD). This change was made to reduce the complexity of the aggregation pipeline of the Get Reports.

Get reports

To fulfill the Get Reports operation, one aggregation pipeline is required:

const pipeline = [
  { $match: docsFromKeyBetweenDate },
  { $addFields: buildTotalsField },
  { $group: groupCountTotals },
  { $project: format },
];

This aggregation operation has a similar logic to the one in appV6R1, with the only differences being the implementation in the $addFields stage.

{ $addFields: buildTotalsField }

• It follows a similar logic to the previous revision, where we first convert the items document into an array using $objectToArray, and then use the reduce function to iterate over the array, accumulating the status.

• The difference lies in the initial value and the logic of the reduce function.

• The initial value in this case is an object/document with one field for each of the report date ranges. These fields for each report date range are also an object/document, with their fields being the possible status set to zero, as this is the initial value.

• The logic in this case checks the date range of the item and increments the totals accordingly. If the item isInOneYearDateRange(...), it is also in all the other date ranges: three, five, seven, and 10 years. If the item isInThreeYearsDateRange(...), it is also in all the other wide date ranges, five, seven, and 10 years.

• The following JavaScript code is logic equivalent to the real aggregation pipeline code. Senior developers could make the argument that this implementation could be less verbose or more optimized. However, due to how MongoDB aggregation pipeline operators are specified, this is how it was implemented.

const itemsArray = Object.entries(items); // Convert the object to an array of [key, value]

const totals = itemsArray.reduce(
  (totals, [YYYYMMDD, status]) => {
  const [YYYY] = YYYYMMDD.slice(0, 4).toString(); // Get year
  const [MM] = YYYYMMDD.slice(4, 6).toString(); // Get month
  const [DD] = YYYYMMDD.slice(6, 8).toString(); // Get day
    let statusDate = new Date(`${YYYY}-${MM}-${DD}`);

    if isInOneYearDateRange(statusDate) {
      totals.oneYear = incrementTotals(totals.oneYear, status);
      totals.threeYears = incrementTotals(totals.threeYears, status);
      totals.fiveYears = incrementTotals(totals.fiveYears, status);
      totals.sevenYears = incrementTotals(totals.sevenYears, status);
      totals.tenYears = incrementTotals(totals.tenYears, status);
    } else if isInThreeYearsDateRange(statusDate) {
      totals.threeYears = incrementTotals(totals.threeYears, status);
      totals.fiveYears = incrementTotals(totals.fiveYears, status);
      totals.sevenYears = incrementTotals(totals.sevenYears, status);
      totals.tenYears = incrementTotals(totals.tenYears, status);
    } else if isInFiveYearsDateRange(statusDate) {
      totals.fiveYears = incrementTotals(totals.fiveYears, status);
      totals.sevenYears = incrementTotals(totals.sevenYears, status);
      totals.tenYears = incrementTotals(totals.tenYears, status);
    } else if isInSevenYearsDateRange(statusDate) {
      totals.sevenYears = incrementTotals(totals.sevenYears, status);
      totals.tenYears = incrementTotals(totals.tenYears, status);
    } else if isInTenYearsDateRange(statusDate) {
      totals.tenYears = incrementTotals(totals.tenYears, status);
    }

    return totals;
  },
  {
    oneYear: { a: 0, n: 0, p: 0, r: 0 },
    threeYears: { a: 0, n: 0, p: 0, r: 0 },
    fiveYears: { a: 0, n: 0, p: 0, r: 0 },
    sevenYears: { a: 0, n: 0, p: 0, r: 0 },
    tenYears: { a: 0, n: 0, p: 0, r: 0 },
  },
);

Indexes

No additional indexes are required, maintaining the single _id index approach established in the appV4 implementation.

Initial scenario statistics

Collection statistics

To evaluate the performance of appV6R3, we inserted 500 million event documents into the collection using the schema and Bulk Upsert function described earlier. For comparison, the tables below also include statistics from previous comparable application versions:

Event statistics

To evaluate the storage efficiency per event, the Event Statistics are calculated by dividing the total data size and index size by the 500 million events.

Because we are adding the year (YYYY) information in the name of each items document field, we got a 16.3% increase in storage size when compared to appV6R1 and a 4.8% increase in storage size when compared to appV6R2. This increase in storage size may be compensated by the gains in the Get Reports function, as we saw when going from appV6R1 to appV6R2.

Load test results

Executing the load test for appV6R3 and plotting it alongside the results for appV6R2, we have the following results for Get Reports and Bulk Upsert.

Get Reports rate

We achieved a significant improvement by transitioning from appV6R2 to appV6R3. For the first time, the application successfully reached all the desired rates in a single phase.

Get Reports latency

The latency saw significant improvements, with the peak value reduced by 71% in the first phase, 67% in the second phase, 47% in the third phase, and 30% in the fourth phase.

Bulk Upsert rate

As had happened in the previous version, the application was able to reach all the desired rates.

Bulk Upsert latency

Here, we have one of the most significant gains in this series: The latency has decreased from seconds to milliseconds. We went from a peak of 1.8 seconds to 250ms in the first phase, from 2.3 seconds to 400ms in the second phase, from 2 seconds to 600ms in the third phase, and from 2.2 seconds to 800ms in the fourth phase.

Issues and improvements

The main bottleneck in our MongoDB server is still the disk throughput. As mentioned in the previous Issues and Improvements, this was the application-level improvement. How can we further optimize on our current hardware?

If we take a closer look at the MongoDB documentation, we'll find out that by default, it uses block compression with the snappy compression library for all collections. Before the data is written to disk, it'll be compressed using the snappy library to reduce its size and speed up the writing process.

Would it be possible to use a different and more effective compression library to reduce the size of the data even further and, as a consequence, reduce the load on the server's disk? Yes, and in the following application revision, we will use the zstd compression library instead of the default snappy compression library.

Application version 6 revision 4 (appV6R4)

As discussed in the previous Issues and Improvements section, the performance gains of this version will be provided by changing the algorithm of the collection block compressor. By default, MongoDB uses the snappy, which we will change to zstd to achieve a better compression performance at the expense of more CPU usage.

All the schemas, functions, and code from this version are exactly the same as the appV6R3.

To create a collection that uses the zstd compression algorithm, the following command can be used.

db.createCollection("<collection-name>", {
  storageEngine: { wiredTiger: { configString: "block_compressor=zstd" } },
});

Schema

The application implementation presented above would have the following TypeScript document schema denominated SchemaV6R0:

export type SchemaV6R0 = {
  _id: Buffer;
  items: Record<
    string,
    {
      a?: number;
      n?: number;
      p?: number;
      r?: number;
    }
  >;
};

Bulk upsert

Based on the specifications, the following bulk updateOne operation is used for each event generated by the application:

const YYYYMMDD = getYYYYMMDD(event.date); // Extract the year(YYYY), month(MM), and day(DD) from the `event.date`

const operation = {
  updateOne: {
    filter: { _id: buildId(event.key, event.date) }, // key + year + quarter
    update: {
      $inc: {
        [`items.${YYYYMMDD}.a`]: event.approved,
        [`items.${YYYYMMDD}.n`]: event.noFunds,
        [`items.${YYYYMMDD}.p`]: event.pending,
        [`items.${YYYYMMDD}.r`]: event.rejected,
      },
    },
    upsert: true,
  },
};

This updateOne is exactly the same logic as the one for appV6R3.

Get reports

Based on the information​​ presented in the Introduction, we have the following aggregation pipeline to generate the reports document.

const pipeline = [
  { $match: docsFromKeyBetweenDate },
  { $addFields: buildTotalsField },
  { $group: groupCountTotals },
  { $project: format },
];

This pipeline is exactly the same logic as the one for appV6R3.

Indexes

No additional indexes are required, maintaining the single _id index approach established in the appV4 implementation.

Initial scenario statistics

Collection statistics

To evaluate the performance of appV6R4, we inserted 500 million event documents into the collection using the schema and Bulk Upsert function described earlier. For comparison, the tables below also include statistics from previous comparable application versions:

Event statistics

To evaluate the storage efficiency per event, the Event Statistics are calculated by dividing the total data size and index size by the 500 million events.

Since the application implementation of appV6R4 is the same as appV5R3, the values for Data Size, Document Size, and Index Size remain the same. The difference lies in Storage Size, which represents the Data Size after compression. Going from snappy to zstd decreased the Storage Size a jaw-dropping 43%. Looking at the Event Statistics, there was a reduction of 26% of the storage required to register each event, going from 8.1 bytes to 6 bytes. These considerable reductions in size will probably translate to better performance on this version, as our main bottleneck is disk throughput.

Load test results

Executing the load test for appV6R4 and plotting it alongside the results for appV6R3, we have the following results for Get Reports and Bulk Upsert.

Get Reports rate

Although we didn't achieve all the desired rates, we saw a significant improvement from appV6R3 to appV6R4. This revision allowed us to reach the desired rates in the first, second, and third quarters.

Get Reports latency

The latency also saw significant improvements, with the peak value reduced by 30% in the first phase, 57% in the second phase, 61% in the third phase, and 57% in the fourth phase.

Bulk Upsert rate

As had happened in the previous version, the application was able to reach all the desired rates.

Bulk Upsert latency

Here, we also achieved considerable improvements, with the peak value being reduced by 48% in the first phase, 39% in the second phase, 43% in the third phase, and 47% in the fourth phase.

Issues and improvements

Although this is the final version of the series, there is still room for improvement. For those willing to try them by themselves, here are the ones that I was able to think of:

• Use the Computed Pattern in the appV6R4.

• Optimize the aggregation pipeline logic for Get Reports in the appV6R4.

• Change the zstd compression level from its default value of 6 to a higher value.

Conclusion

This final part of "The Cost of Not Knowing MongoDB" series has explored the ultimate evolution of MongoDB application optimization, demonstrating how revolutionary design patterns and infrastructure-level improvements can transcend traditional performance boundaries. The journey through appV6R0 to appV6R4 represents the culmination of sophisticated MongoDB development practices, achieving performance levels that seemed impossible with the baseline appV1 implementation.

Series transformation summary

From foundation to revolution: The complete series showcases a remarkable transformation across three distinct optimization phases.

• Part 1 (appV1-appV4): Document-level optimizations achieving 51% storage reduction through schema refinement, data type optimization, and strategic indexing.

• Part 2 (appV5R0-appV5R4): Advanced pattern implementation with the Bucket and Computed Patterns, delivering 89% index size reduction and first-time achievement of target rates.

• Part 3 (appV6R0-appV6R4): Revolutionary Dynamic Schema Pattern with infrastructure optimization, culminating in sub-second latencies and comprehensive target rate achievement.

Performance evolution: The progression reveals exponential improvements across all metrics.

• Get Reports latency: From 6.5 seconds (appV1) to 200-800ms (appV6R4)—a 92% improvement.

• Bulk Upsert latency: From 62 seconds (appV1) to 250-800ms (appV6R4)—a 99% improvement.

• Storage efficiency: From 128.1B per event (appV1) to 6.0B per event (appV6R4)—a 95% reduction.

• Target rate achievement: From consistent failures to sustained success across all operational phases.

Architectural paradigm shifts

The Dynamic Schema Pattern revolution: appV6R0 through appV6R4 introduced the most sophisticated MongoDB design pattern explored in this series. The Dynamic Schema Pattern fundamentally redefined data organization by

• Eliminating array overhead: Replacing MongoDB arrays with computed object structures to minimize storage and processing costs.

  • Single-pipeline optimization: Consolidating five separate aggregation pipelines into one optimized operation, reducing computational overhead by 80%.

• Infrastructure-level optimization: Implementing zstd compression, achieving 43% additional storage reduction over default snappy compression.

Query optimization breakthroughs: The implementation of intelligent date range calculation within aggregation pipelines eliminated redundant operations while maintaining data accuracy. This approach demonstrates senior-level MongoDB development by leveraging advanced aggregation framework capabilities to achieve both performance and maintainability.

Critical technical insights

Performance bottleneck evolution: Throughout the series, we observed how optimization focus shifted as bottlenecks were resolved

1. Initial phase: Index size and query inefficiency dominated performance.

2. Intermediate phase: Document retrieval count became the limiting factor.

3. Advanced phase: Aggregation pipeline complexity constrained throughput.

4. Final phase: Disk I/O emerged as the ultimate hardware limitation.

Pattern application maturity: The series demonstrates the progression from junior to senior MongoDB development practices

• Junior level: Schema design without understanding indexing implications (appV1)

• Intermediate level: Applying individual optimization techniques (appV2-appV4)

• Advanced level: Implementing established MongoDB patterns (appV5RX)

• Senior level: Creating custom patterns and infrastructure optimization (appV6RX)

Production implementation guidelines

When to apply each pattern: Based on the comprehensive analysis, the following guidelines emerge for production implementations

• Document-level optimizations: Essential for all MongoDB applications, providing 40-60% improvement with minimal complexity

• Bucket Pattern: Optimal for time-series data with 10:1 or greater read-to-write ratios

• Computed Pattern: Most effective in read-heavy scenarios with predictable aggregation requirements

• Dynamic Schema Pattern: Reserved for high-performance applications where development complexity trade-offs are justified

Infrastructure considerations: The zstd compression implementation in appV6R4 demonstrates that infrastructure-level optimizations can provide substantial benefits (40%+ storage reduction) with minimal application changes. However, these optimizations require careful CPU utilization monitoring and may not be suitable for CPU-constrained environments.

The true cost of not knowing MongoDB

This series reveals that the "cost" extends far beyond mere performance degradation:

Quantifiable impacts:

• Resource utilization: Up to 20x more storage requirements for equivalent functionality

• Infrastructure costs: Potentially 10x higher hardware requirements due to inefficient patterns

• Developer productivity: Months of optimization work that could be avoided with proper initial design

• Scalability limitations: Fundamental architectural constraints that become exponentially expensive to resolve

Hidden complexities: More critically, the series demonstrates that MongoDB's apparent simplicity can mask sophisticated optimization requirements. The transition from appV1 to appV6R4 required a deep understanding of

• Aggregation framework internals and optimization strategies.

• Index behavior with different data types and query patterns.

• Storage engine compression algorithms and trade-offs.

• Memory management and cache utilization patterns.

Final recommendations

For development teams:

1. Invest in MongoDB education: The performance differences documented in this series justify substantial training investments.

2. Establish pattern libraries: Codify successful patterns like those demonstrated to prevent anti-pattern adoption.

3. Implement performance testing: Regular load testing reveals optimization opportunities before they become production issues.

4. Plan for iteration: Schema evolution is inevitable; design systems that accommodate architectural improvements.

For architectural decisions:

1. Start with fundamentals: Proper indexing and schema design provide the foundation for all subsequent optimizations.

2. Measure before optimizing: Each optimization phase in this series was guided by comprehensive performance measurement.

3. Consider total cost of ownership: The development complexity of advanced patterns must be weighed against performance requirements.

4. Plan infrastructure scaling: Understanding that hardware limitations will eventually constrain software optimizations.

Closing reflection

The journey from appV1 to appV6R4 demonstrates that MongoDB mastery requires understanding not just the database itself, but the intricate relationships between schema design, query patterns, indexing strategies, aggregation frameworks, and infrastructure capabilities. The 99% performance improvements documented in this series are achievable, but they demand dedication to continuous learning and sophisticated engineering practices.

For organizations serious about MongoDB performance, this series provides both a roadmap for optimization and a compelling case for investing in advanced MongoDB expertise. The cost of not knowing MongoDB extends far beyond individual applications—it impacts entire technology strategies and competitive positioning in data-driven markets.

The patterns, techniques, and insights presented throughout this three-part series offer a comprehensive foundation for building high-performance MongoDB applications that can scale efficiently while maintaining operational excellence. Most importantly, they demonstrate that with proper knowledge and application, MongoDB can deliver extraordinary performance that justifies its position as a leading database technology for modern applications.

In all seriousness, legacy systems and technical debt are huge challenges: the cost of tech debt has been estimated at almost $4 trillion dollars. That’s trillion with a T! Legacy systems can slow down innovation, create bottlenecks, and make it tough to deliver the seamless, real-time experiences customers increasingly expect. But companies are finding that modernizing their applications isn't just about fixing what's broken—modernization enables them to move faster and innovate for end-users.

That’s why I'm incredibly excited to share the recent launch of MongoDB’s Application Modernization Platform (AMP). This AI-powered program is designed to help enterprises move beyond outdated infrastructures to embrace a flexible, data-driven future. AMP is a comprehensive approach to modernization that combines smart AI tooling with proven methodologies, enabling businesses to transform their applications from the ground up, moving from legacy monoliths to a more flexible, microservices-based architecture.

In this roundup, we're spotlighting customers who understand the strategic importance of modernization. You'll see how Wells Fargo is using MongoDB to power a new credit card platform, how CSX is ensuring business continuity during a critical migration, how Intellect Design is modernizing its wealth management platform, and how Deutsche Telekom is transforming its B2C digital channels. With MongoDB, customers are showing how integral a modern database is to powering the next generation of applications—and succeeding in the AI era.

Wells Fargo

Wells Fargo sought to modernize its mainframe-dependent credit card platform to provide a faster, more seamless customer experience and handle an exponential increase in transaction data. The company's legacy system was costly to manage and lacked the scalability needed for its "Cards 2.0" initiative.

To solve this, Wells Fargo built an operational data store (ODS) using MongoDB. This new platform allowed them to adopt reusable APIs, streamline integrations, and move from a monolithic architecture to flexible microservices. The ODS now serves 40% of traffic from external vendors, handling more than 7 million transactions with sub-second service.

By leveraging MongoDB, Wells Fargo was able to jumpstart its mainframe modernization and create curated data products to serve real-time, personalized financial services.

CSX

CSX, a major U.S. railroad company, sought to modernize its critical operations platform, RTOP, by migrating it to the cloud. The challenge was to maintain the platform's 24/7 availability with minimal disruption to its mission-critical, near real-time operations during the transition.

To solve this, CSX selected MongoDB Atlas on Azure and partnered with MongoDB Professional Services. Leveraging the Cluster-to-Cluster Sync (mongosync) feature, the team was able to facilitate continuous data synchronization and complete the entire migration in just a few hours.

The move to MongoDB Atlas has equipped CSX with a more scalable and resilient platform. This modernization effort established a blueprint for migrating other critical applications and helped CSX continue its digital transformation journey toward becoming America’s best-run railroad.

Intellect Design

Intellect Design, a global fintech company, sought to modernize its wealth management platform to overcome legacy system bottlenecks and multihour batch processing delays. The company's rigid relational database architecture limited its ability to scale and innovate.

To solve this, the company partnered with MongoDB, using our AMP methodology and generative AI tools. This transformation reengineered the platform's core components, resulting in an 85% reduction in onboarding workflow times, allowing clients to access critical portfolio insights faster than ever.

This initiative is the first step in Intellect Design's long-term vision to integrate its entire application suite into a unified, AI-driven service. By leveraging MongoDB Atlas's flexible schema and powerful native tools, the company is now better positioned to deliver smarter analytics and advanced AI capabilities to its customers.

Deutsche Telekom

Deutsche Telekom, a leading telecommunications company, sought to modernize its B2C digital channels, which were fragmented by outdated legacy systems. The company needed to create a unified digital experience for its 30 million customers while improving developer productivity.

By leveraging MongoDB Atlas as part of its Internal Developer Platform, Deutsche Telekom built a robust data infrastructure to unify customer data and power its new digital services. This approach allowed the company to retire legacy systems and reduce its reliance on physical shops and call centers.

The transition to MongoDB Atlas led to a massive surge in digital engagement, with daily customer interactions rising from under 50,000 to approximately 1.5 million. The company's customer data platform now handles up to 15 times the load of legacy systems, supporting large-scale loyalty programs and transforming the customer experience.

Video spotlight: Bendigo Bank

Before you go, watch how Bendigo and Adelaide Bank modernized their core banking technology using MongoDB Atlas and generative AI.

But I felt like something was missing.

It felt clunky. My queries were longer than I expected, and performance wasn’t great; I had the sense that I was fighting with the database instead of working with it. After a few projects like that, I began to wonder if maybe MongoDB wasn’t for me.

Looking back now, I can say the problem wasn’t MongoDB, but was somewhere between the keyboard and the chair. It was me. I was carrying over habits from years of working with relational databases, expecting the same rules to apply.

If MongoDB’s Skill Badges had existed when I started, I think my learning curve would have been a lot shorter. I had to learn many lessons the hard way, but these new badges cover the skills I had to piece together slowly. Instead of pretending I nailed it from day one, here’s the honest version of how I learned MongoDB, what tripped me up along the way, and how these Skill Badges would have helped.

Learning to model the MongoDB way

The first thing I got wrong was data modeling. I built my schema like I was still working in SQL– every entity in its own collection, always referencing instead of embedding, and absolutely no data duplication. It felt safe because it was familiar.

Then I hit my first complex query. It required data from various collections, and suddenly, I found myself writing a series of queries and stitching them together in my code. It worked, but it was a messy process.

When I discovered embedding, it felt like I had found a cheat code. I could put related data together in one single document, query it in one shot, and get better performance.

That’s when I made my second mistake. I started embedding everything.

At first, it seemed fine. However, my documents grew huge, updates became slower, and I was duplicating data in ways that created consistency issues. That’s when I learned about patterns like Extended References, and more generally, how to choose between embedding and referencing based on access patterns and update frequency.

Later, I ran into more specialized needs, such as pre-computing data, embedding a subset of a large dataset into a parent, and tackling schema versioning. Back then, I learned those patterns by trial and error. Now, they’re covered in badges like Relational to Document Model, Schema Design Patterns, and Advanced Schema Patterns.

Fixing what I thought was “just a slow query”

Even after I got better at modeling, performance issues kept popping up. One collection in particular started slowing down as it grew, and I thought, “I know what to do! I’ll just add some indexes.”

I added them everywhere I thought they might help. Nothing improved.

It turns out indexes only help if they match your query patterns. The order of fields matters, and whether you cover your query shapes will affect performance. Most importantly, just because you can add an index doesn’t mean that you should be adding it in the first place.

The big shift for me was learning to read an explain() plan and see how MongoDB was actually executing my queries. Once I started matching my indexes to my queries, performance went from “ok” to “blazing fast.”

Around the same time, I stopped doing all my data transformation in application code. Before, I’d pull in raw data and loop through it to filter, group, and calculate. It was slow, verbose, and easy to break. Learning the aggregation framework completely changed that. I could handle the filtering and grouping right in the database, which made my code cleaner and the queries faster.

There was a lot of guesswork in how I created my indexes, but the new Indexing Design Fundamentals covers that now. And when it comes to querying and analyzing data, Fundamentals of Data Transformation is there to help you. Had I had those two skills when I first started, I would’ve saved a lot of time wasted on trial and error.

Moving from “it works” to “it works reliably”

Early on, my approach to monitoring was simple: wait for something to break, then figure out why. If a performance went down, I’d poke around in logs. If a server stopped responding, I’d turn it off and on again, and hope for the best.

It was stressful, and it meant I was always reacting instead of preventing problems.

When I learned to use MongoDB’s monitoring tools properly, that changed. I could track latency, replication lag, and memory usage. I set alerts for unusual query patterns. I started seeing small problems before they turned into outages.

Performance troubleshooting became more methodical as well. Instead of guessing, I measured. Breaking down queries, checking index use, and looking at server metrics side by side. The fixes were faster and more precise.

Reliability was the last piece I got serious about. I used to think a working cluster was a reliable cluster. But reliability also means knowing what happens if a node fails, how quickly failover kicks in, and whether your recovery plan actually works in practice.

Those things you can now learn in the Monitoring Tooling, Performance Tools and Techniques, and Cluster Reliability skill badges. If you are looking at deploying and maintaining MongoDB clusters, these skills will teach you what you need to know to make your deployment more resilient.

Getting curious about what’s next

Once my clusters were stable, I stopped firefighting, and my mindset changed. When you trust your data model, your indexes, your aggregations, and your operations, you get to relax. You can then spend that time on what’s coming next instead of fixing what’s already in production.

For me, that means exploring features I wouldn’t have touched earlier, like Atlas Search, gen AI, and Vector Search. Now that the fundamentals are solid, I can experiment without risking stability and bring in new capabilities when a project actually calls for them.

What I’d tell my past self

If I could go back to when I first installed MongoDB, I’d keep it simple:

• Focus on data modeling first. A good foundation will save you from most of the problems I ran into.

• Once you have that, learn indexing and aggregation pipelines. They will make your life much easier when querying.

• Start monitoring from day one. It will save you a lot of trouble in the long run.

• Take a moment to educate yourself. You can only learn so much from trial and error. MongoDB offers a myriad of resources and ways to upskill yourself.

Once you have established that base, you can explore more advanced topics and uncover the full potential of MongoDB. Features like Vector Search, full-text search with Atlas Search, or advanced schema design patterns are much easier to adopt when you trust your data model and have confidence in your operational setup.

What if your search box didn’t just find results, but instead understood intent? That’s exactly what this integration delivers!

Beyond search: From matching to meaning

Developers rely on MongoDB’s expansive knowledge ecosystem to find answers fast. But even with a rich library of technical blogs, forum threads, and documentation, traditional keyword search often falls short—especially when queries are nuanced, multilingual, or context-driven.

That’s where the MongoDB-Pureinsights solution shines. Built on MongoDB Atlas and orchestrated by the Pureinsights Discovery platform, this intelligent search experience starts with the fundamentals: fast, accurate keyword results, powered by MongoDB Atlas Search.

But as queries grow more ambiguous—say, “tutorials for AI”—the platform steps up. MongoDB Atlas Vector Search with Voyage AI, available as an embedding and reranking option (now part of MongoDB), goes beyond literal keywords to interpret intent—helping applications deliver smarter, more relevant results. The outcome: smarter, semantically aware responses that feel intuitive and accurate—because they are.

What’s more, with generative answers enabled, the platform synthesizes information across MongoDB’s ecosystem (blog content, forums, and technical docs) to deliver clear, contextual answers using state-of-the-art language models. But it's not just pointing you to the right page. Instead, the platform is providing the right answer, with citations, ready to use. It’s like embedding a domain-trained AI assistant directly into your search bar.

“As organizations look to move beyond traditional keyword search, they need solutions that combine speed, relevance, and contextual understanding,” said Haim Ribbi, Vice President, Global CSI, VAR & Tech Partner at MongoDB. “MongoDB Atlas provides the foundation for smarter discovery, and this collaboration with Pureinsights shows how easily teams can deliver gen AI-powered search experiences using their existing content.”

Built for users everywhere

But intelligence alone doesn’t make it transformational. What sets this experience apart is its adaptability. Whether you’re a developer troubleshooting in Berlin or a product owner building in São Paulo, the platform tailors responses to your preferences.

Prefer concise summaries or deep technical dives? Want to translate answers in real time? Need responses that reflect your role and context? You’re in control. From tone and length to language and specificity, this is a search that truly understands you—literally and figuratively.

Built on MongoDB. Elevated by Voyage AI. Delivered by Pureinsights.

At the core of this solution is MongoDB Atlas, which unifies fast, scalable data access to structured content through Atlas Search and Atlas Vector Search. Looking ahead, by integrating with Voyage AI’s industry-leading embedding models, MongoDB Atlas aims to make semantic search and retrieval-augmented generation (RAG) applications even more accurate and reliable. While currently in private preview, this enhancement signals a promising future for developers building intelligent, AI-powered experiences.

Pureinsights handles the orchestration layer. Their Discovery Platform ingests and enriches content, blends keyword, vector, and generative search into a seamless UI, and integrates with large language models like GPT-4. The platform supports multilingual capabilities, easy deployment, and enterprise-grade scalability out of the box. While generative answers are powered by integrated large language models (LLMs) and may vary by deployment, the solution is enterprise-ready, cloud-native, and built to scale.

Bringing intelligent discovery to your own data

Watch the demo video to see AI-powered search in action across 4,000+ pages of MongoDB content—from community forums and blog posts to technical documentation.

While the demo features MongoDB’s content, the solution is built to adapt. You can bring the same AI-powered experience to your internal knowledge base, customer support portal, or developer hub—no need to build from scratch.

Is keyword search no longer useful? While the rise of LLMs and vector search may suggest that traditional keyword search is becoming less prevalent, the future of search actually relies on effectively combining both methods. This is where hybrid search plays a crucial role, blending the precision of traditional text search with the powerful contextual understanding of vector search. Despite advances in vector technology, keyword search still has a lot to contribute and remains essential to meeting current user expectations.

The rise of hybrid search

By late 2022 and particularly throughout 2023, as vector search saw a surge in popularity (see image 1 below), it quickly became clear that vector embeddings alone were not enough. Even as embedding models continue to improve at retrieval tasks, full-text search will always remain useful for identifying tokens outside the training corpus of an embedding model. That is why users soon began to combine vector search with lexical search, exploring ways to leverage both precision and context-aware retrieval. This shift was driven in large part by the rise of generative AI use cases like retrieval-augmented generation (RAG), where high-quality retrieval is essential.

As hybrid search matured beyond basic score combination, the main fusion techniques emerged - reciprocal rank fusion (RRF) and relative score fusion (RSF). They offer ways to combine results that do not rely on directly comparable score scales. RRF focuses on ranking position, rewarding documents that consistently appear near the top across different retrieval methods. RSF, on the other hand, works directly with raw scores from different sources of relevance, using normalization to minimize outliers and align modalities effectively at a more granular level than rank alone can provide. Both approaches quickly gained traction and have become standard techniques in the market.

How did the market react?

The industry realized the need to introduce hybrid search capabilities, which brought different challenges for different types of players.

For lexical-first search platforms, the main challenge was to add vector search features and implement the bridging logic with their existing keyword search infrastructure. These vendors understood that the true value of hybrid search emerges when both modalities are independently strong, customizable, and tightly integrated.

On the other hand, vector-first search platforms faced the challenge of adding lexical search. Implementing lexical search through traditional inverted indexes was often too costly due to storage differences, increased query complexity, and architectural overhead. Many adopted sparse vectors, which represent keyword importance in a way similar to traditional term-frequency methods used in lexical search. Sparse vectors were key for vector-first databases in enabling a fast integration of lexical capabilities without overhauling the core architecture.

Hybrid search soon became table stakes and the industry focus shifted toward improving developer efficiency and simplifying integration. This led to a growing trend of vendors building native hybrid search functions directly into their platforms. By offering out-of-the-box support to combine and manage both search types, the delivery of powerful search experiences was accelerated.

As hybrid search became the new baseline, more sophisticated re-ranking approaches emerged. Techniques like cross-encoders, learning-to-rank models, and dynamic scoring profiles began to play a larger role, providing systems with additional alternatives to capture nuanced user intent. These methods complement hybrid search by refining the result order based on deeper semantic understanding.

What to choose? Lexical-first or vector-first solutions? Top considerations when choosing a hybrid search solution

When choosing how to implement hybrid search, your existing infrastructure plays a major role in the decision. For users working within a vector-first database, leveraging their lexical capabilities without rethinking the architecture is often enough. However, if the lexical search requirements are advanced, commonly the optimal solution is served with a traditional lexical search solution coupled with vector search, like MongoDB. Traditional lexical - or lexical-first - search offers greater flexibility and customization for keyword search, and when combined with vectors, provides a more powerful and accurate hybrid search experience.

Indexing strategy is another factor to consider. When setting up hybrid search, users can either keep keyword and vector data in separate indexes or combine them into one. Separate indexes give more freedom to tweak each search type, scale them differently, and experiment with scoring. The compromise is higher complexity, with two pipelines to manage and the need to normalize scores. On the other hand, a combined index is easier to manage, avoids duplicate pipelines, and can be faster since both searches run in a single pass. However, it limits flexibility to what the search engine supports and ties the scaling of keyword and vector search together. The decision is mainly a trade-off between control and simplicity.

Lexical-first solutions were built around inverted indexes for keyword retrieval, with vector search added later as a separate component. This often results in hybrid setups that use separate indexes. Vector-first platforms were designed for dense vector search from the start, with keyword search added as a supporting feature. These tend to use a single index for both approaches, making them simpler to manage but sometimes offering less mature keyword capabilities.

Lastly, a key aspect to take into account is the implementation style. Solutions with hybrid search functions handle the combination of lexical and vector search natively, removing the need for developers to manually implement it. This reduces development complexity, minimizes potential errors, and ensures that result merging and ranking are optimized by default. Built-in function support streamlines the entire implementation, allowing teams to focus on building features rather than managing infrastructure.

In general, lexical-first systems tend to offer stronger keyword capabilities and more flexibility in tuning each search type, while vector-first systems provide a simpler, more unified hybrid experience. The right choice depends on whether you prioritize control and mature lexical features or streamlined management with lower operational overhead.

How does MongoDB do it?

When vector search emerged, MongoDB added vector search indexes to the existing traditional lexical search indexes. With that, MongoDB evolved into a competitive vector database by providing developers with a unified architecture for building modern applications. The result is an enterprise-ready platform that integrates traditional lexical search indexes and vector search indexes into the core database.

MongoDB recently released native hybrid search functions to MongoDB Atlas and as part of a public preview for use with MongoDB Community Edition and MongoDB Enterprise Server deployments. This feature is part of MongoDB’s integrated ecosystem, where developers get an out-of-the-box hybrid search experience to enhance the accuracy of application search and RAG use cases.

As a result, instead of managing separate systems for different workloads, MongoDB users benefit from a single platform designed to support both operational and AI-driven use cases. As generative AI and modern applications advance, MongoDB gives organizations a flexible, AI-ready foundation that grows with them.

This inconsistency creates friction, wastes resources, and ultimately, exposes businesses to unnecessary risk.

At MongoDB, we believe that securing the SaaS ecosystem is a shared responsibility. That's why we were proud to collaborate with the Cloud Security Alliance (CSA) and industry leaders like GuidePoint Security to develop a new standard—the SaaS Security Capability Framework (SSCF).

The problem: A gap in cloud security

For years, the majority of security assessments have focused on the SaaS provider's organizational security, often through frameworks like SOC 2 or ISO 27001. While essential, these frameworks don't always address a critical question: what security capabilities are available to the SaaS customer within the application?

This gap means that security teams face a chaotic landscape. Every new SaaS app brings a different set of configurable controls for logging, identity management, and data access. This makes it nearly impossible to implement and track consistent security policies at scale, leading to a burdensome assessment process for everyone involved.

The solution: A common framework for SaaS security

The SSCF was created to solve this problem by establishing a clear, technical set of customer-facing security controls that SaaS vendors should provide. The framework is designed to empower customers by ensuring they have the tools they need to operate applications securely at scale on their side of the Shared Security Responsibility Model (SSRM).

The framework helps with many use cases, but three key audiences stand out:

• For risk management teams: The SSCF provides a clear baseline to use during vendor assessments, simplifying procurement.

• For SaaS security teams: It offers a checklist for implementing the security features enterprises expect, streamlining the security program.

• For SaaS vendors: The SSCF standardizes assessment responses, reducing the overhead of custom questionnaires and helping vendors meet customer requirements.

The SSCF focuses on six critical domains, aligned with CSA’s Cloud Control Matrix, providing specific and actionable controls for each:

1. Change Control and Configuration Management (CCC): Ensuring you can programmatically query and get documentation on all security configurations.

2. Data Security and Privacy Lifecycle Management (DSP): Giving customers control over features like disabling file uploads to prevent malicious code.

3. Identity and Access Management (IAM): Providing robust, modern controls for user access, including SSO enforcement, non-human identity (NHI) governance, and a dedicated read-only security auditor role.

4. Interoperability and Portability (IPY): Giving administrators control over mass data exports and visibility into application integrations.

5. Logging and Monitoring (LOG): Defining a clear set of comprehensive requirements for machine-readable logs with mandatory fields for effective threat detection and forensics.

6. Security Incident Management (SEF): Requiring a simple, effective way for vendors to notify a designated customer security contact during an incident.

MongoDB's commitment to a more secure ecosystem

Our involvement in creating the SSCF stems from our deep commitment to the security of our customers' data and the broader developer community. We believe that robust security shouldn't be an afterthought; it must be built in and easy to consume. The principles outlined in the SSCF—like strong identity controls and comprehensive logging—are philosophies we already built into our own data platform.

Strong security capabilities allow our customers to build and innovate faster and more securely, knowing they have a reliable foundation. And personally, as a co-chair of the CSA SSCF, I’ve seen great excitement and engagement on the part of our working group—which helped me realize how many companies are affected by this lack of consistency.

The SSCF is a vital step toward creating a more trusted, efficient, and secure global SaaS ecosystem. We are thrilled to have been a part of this foundational work and will continue to champion this standard that empowers developers and security teams alike.

We’re relatively new to MongoDB—Ashish joined two years ago via the Granite acquisition after a decade-plus building Google’s databases and distributed systems, and Akshat joined in June 2024 after 15 years building databases at AWS. We have a shared obsession with distributed systems. We’d seen how much developers loved MongoDB, which is part of the reason we joined the company—MongoDB is one of the most loved databases in the world. So one of the first things we sought to understand was why.

It turned out to be simpler than we thought: MongoDB’s vision is to get developers to production fast. This means making it easy to start, and easier to keep going—one command spin-up, sane defaults for day one, and zero downtime upgrades and zero downtime expansion to multiple clouds as you scale. That’s what developer agility looks like in practice: the ability to choose the best tools, move quickly, and to trust the system to carry the weight of failure, complexity, and change.

At MongoDB, three principles drive that vision: resilience, intelligence, and simplicity.

Resilience is the ability to keep going when something breaks, intelligence is the ability to adapt to changing conditions, and simplicity is reducing cognitive and operational load so users and operators can move quickly and safely. These are not just technical goals—we treat them as non-negotiable design constraints. So if a change widens blast radius, breaks adaptive performance, or adds operator toil, it doesn’t ship.

In this post, we share the key engineering themes shaping our work and the mechanisms that keep us honest.

Security as a first principle

Security isn't a wall you build around your data. It's an assumption you design against from the very beginning. The assumption is simple: in a distributed system, you can’t trust the network, you can’t trust the hardware, and you certainly can't trust your neighbors.

This starts with architectural isolation. In most cloud database service offerings, you're sharing walls with strangers. Shared walls hurt performance, they leak failures, and sometimes they leak secrets. We minimize shared walls, and where utilities must be shared, we build firebreaks. Stronger isolation reduces the blast radius of mistakes and attacks. With a MongoDB Atlas dedicated cluster, you get the whole building. Your cluster runs on its own provisioned servers, in its own private network (VPC). Your unencrypted data is never available in a shared VM or process. There are no "noisy neighbors" because you have no neighbors. The attack surface shrinks dramatically, and resource contention disappears. The blast radius of a problem elsewhere stops at your door. In other words, we follow an anti-Vegas principle—what happens outside your cluster will stay outside.

But true security is layered. Once we’ve isolated the environment, we defend it from the inside out. We start by asking the hard questions:

• Who are you? That's strong authentication, from SCRAM to AWS IAM.
• What can you do? That's fine-grained RBAC, enforcing the principle of least privilege.
• What if someone gets in? That's encryption everywhere—in transit, at rest, and even in use with Client-Side Field Level Encryption.
• How do we lock down the roads? That’s network controls like IP access lists and private endpoints.
• And how do we prove it? That's granular auditing for a clear, immutable trail.

Every one of these layers reflects defense in depth.

The history of database security is full of trade-offs between safety and functionality. For decades, the trade-off has been brutal: to run a query, you had to decrypt your data on the server, exposing it to risk. Queryable Encryption—an industry-first searchable encryption scheme developed by MongoDB Research—breaks this paradigm. It allows your application to run expressive queries, including equality and range checks on data that remains fully encrypted on the server. The decryption keys never leave your client. The server maintains encrypted indexes for the fields you wish to query on, and queries can be done entirely on the encrypted data, maintaining the strongest privacy and security of your sensitive data.

By carrying these defenses in the platform itself, security stops being another burden developers have to design around. They get the privacy guarantees, the audit trails, and the compliance, without sacrificing functionality or velocity.

Achieving resilience: Architecture, operations, and proof

Systems don’t live in a vacuum. They live in messy realities: network partitions, power outages, kernel panics, cloud control plane hiccups, operator mistakes. The measure of resilience is not “will it fail?” but “what happens next?” Resilience is the ability to keep going when the thing you depend on stops working, not because you planned for it to fail, but because you planned for it to recover.

Here’s how we achieve resilience.

Architecture: MongoDB Atlas is built on the assumption that something may fail at any time. Every cluster starts life as a replica set, spread across independent availability zones. That’s the default, not an upgrade. The moment a primary becomes unreachable, an election happens. Within seconds, another node takes over, clients reconnect, and in-flight writes retry automatically. Single-zone diversity buys you protection against a data center outage. Adding more regions buys you protection against a full region failure. Adding more cloud providers buys you insulation against provider-wide events. Each step up that ladder buys you more protection against bigger failures. The trade-off is that each step adds more moving parts to manage, and the failure modes evolve: intra-region links are fast; cross-region introduce wide, lossy links; cross-cloud adds different fabrics, load balancers, and failure semantics.

Our job is to make any type of failures (node failures, link failures, gray failures) invisible to you. Writes are only committed when a majority of voting members have the entry in the same term. That rule sounds small, but it’s the safety net that prevents a primary stranded on the wrong side of a partition from accepting writes it can’t keep. Heartbeats and UpdatePosition messages carry progress and truth; if a node learns of a higher term, it steps down immediately. When elections happen, the new primary doesn’t open for writers until it has caught up to the latest known state, preserving as many uncommitted writers as possible. Secondaries apply operations as they arrive, even over lossy links.

Operating discipline: Resilience isn’t just in the code and architecture, it’s in how you operate it every day. Even the best design will fail without the discipline to detect problems early and recover quickly. You need to embed it in how you operate. Operational excellence is about preventing avoidable failures, detecting the ones you can’t prevent, and recovering quickly when they happen.

And we’ve turned that into a discipline. Every week, the people closest to the work—engineers, on-calls, product managers, and leaders—step out of the day’s firefight to review the system with rigor. We celebrate the small wins that quietly make the system safer. We dig into failures to understand not just what happened, but how to make sure it doesn’t happen again anywhere. The goal isn’t perfection. Instead, it’s building a system where every lesson learned and every fix made raises the floor for everyone. A single automation can remove a whole category of incidents. A well-written postmortem can stop the same mistake from happening across dozens of systems. The return isn’t linear—it compounds.

When resilience works, failure stops being something every developer has to carry in their head. The system absorbs it, recovers, and lets them keep moving.

Proof before shipping: Testing tells you that your code works in the cases you’ve thought to test. Formal verification tells you whether it works in all the cases that matter, even the ones you didn’t think to test. MongoDB is among the few cloud databases that apply and publish formal methods on the core database paths. This rigor translates into agility; teams using the database ship products without worrying about node failures, failovers, or clock skew, causing edge cases. Those edge cases in the database have already been explored, proven, and designed against.

When we design a new replication or failover protocol, we don’t just code it, run a few chaos tests, and ship it. We build a mathematical model of the core logic stripped of distracting details like disk format or thread pools and ask a model checker to try every possible interleaving of events. The tool doesn’t skip the “unlikely” cases. It tries them all.

Take logless reconfiguration. The idea is simple: MongoDB decouples configuration changes from the data replication log, so membership changes no longer queue behind user writes. But while the idea is simple, the implementation is not. Without care, concurrent configs can fork the cluster, primaries can be elected on stale terms, or new majorities can lose the old majority’s writes. We modeled the protocol in TLA+, explored millions of interleavings, and distilled the solution down to four invariants: terms block stale primaries, monotonic versions prevent forks, majority votes stop minority splits, and the oplog-commit rule ensures durability carries forward.

For transactions, we developed a modular formal specification of the multi-shard protocol in TLA+ to verify protocol correctness and snapshot isolation, defined and tested the WiredTiger storage interface with automated model-based techniques, and analyzed permissiveness to assess how well concurrency is maximized within the isolation level.

These models are not giant, perfect representations of the whole system. They’re small, precise abstractions that focus on the essence of correctness. The payoff is simple: the model checker explores more corner cases in minutes than a human tester could in years.

Alongside formal proofs, we use additional tools to test the implementation under deterministic simulation: fuzzing, fault injection, and message reordering against real binaries. Determinism gives us one-click bug replication, CI/CD regression gates, and reliable incident replays—o rare timing bugs become easy fixes.

Mastering the multi-cloud reality with simple abstractions

Developer agility isn’t about having a hundred choices on a menu; it's about removing the friction that makes real choice impossible. One such choice that almost never materializes in practice is multi-cloud. We achieve multi-cloud by building a unified data fabric that lets you put your data anywhere you need it, controlled from a single place. A DIY multi-cloud database where you run self-managed MongoDB across AWS, Microsoft Azure, and Google Cloud seems simple on paper. In practice, it involves weeks of networking (VPC/VNet peering, routing, and firewall rules) and brittle scripts. The theoretical agility that you got by going multi-cloud collapses under the weight of operational reality.

Now contrast this with MongoDB Atlas, where you don’t have to manually orchestrate provisioning across three different cloud APIs. A single replica set can span AWS, Google Cloud, and Azure. Provisioning, networking, and failover are handled for you. Your app connects with a standard mongodb+srv string, and our intelligent drivers ensure that if your AWS primary fails, traffic automatically fails over to a new primary in GCP or Azure without any changes to your code. This transforms an operational nightmare into a simple deployment choice, giving you freedom from vendor lock-in and a robust defense against provider-wide outages.

Agility also means precise data placement for data sovereignty and global latency. Global Clusters and Zone sharding let you describe simple rules so data stays where policy requires and users are served locally, e.g., A rule to map "DE", "FR", and "ES" to the EU_Zone can guarantee that all European customer data and order history physically reside within European borders, satisfying strict GDPR requirements out of the box. Because Zone Sharding is built into the core sharding system, you can add or adjust placement without app rewrites. That’s real agility: the platform removes the hard parts, so the choices are real.

From data to intelligence: Building the next generation of AI-powered applications

Building intelligent AI-powered features has been a complex and fragmented process. The traditional approach forced developers to maintain separate vector databases for semantic search, creating brittle ETL pipelines to shuttle data back and forth from their primary operational database. This introduced architectural complexity, latency, and a higher total cost of ownership. That’s not agility. That’s friction.

Our approach is to eliminate this friction entirely. We believe the best place to build AI-powered applications is directly on your operational data. This is the vision behind MongoDB Atlas Vector Search. Instead of creating a separate product, we integrated vector search capabilities directly into the MongoDB query engine. This is a profound simplification for developers. You can now perform semantic search—finding results based on meaning and context, not just keywords—using the same MongoDB Query API (MQL) and drivers you already know. There are no new systems to learn and no data to synchronize. You can seamlessly combine vector search with traditional filters, aggregations, and updates in a single, expressive query. This dramatically accelerates the development of modern features like RAG (retrieval-augmented generation) for chatbots, sophisticated recommendation engines, and intelligent search experiences. Intelligence isn’t something you bolt on. It’s something you build on.

This is an area where we continue to make multiple enhancements. For example, with the acquisition of Voyage AI earlier this year, we are making progress towards integrating Voyage's embedding and reranking models into Atlas to deliver a truly native experience. We are also actively applying AI toward our Application Modernization efforts. Consider a relational database application that involves pages of SQL statements representing a view or a query. How do you translate it so it can work effectively with MongoDB’s MQL? LLMs have advanced enough to provide a base version that may be mostly the correct shape, but to get it accurate and performant requires building additional tooling. We are actively working with several customers, not only on the SQL → MQL translation, but also on modernizing their application code using similar techniques.

What’s next?

We’ll keep pushing on the same three levers: resilience, intelligence, and simplicity. Keep watching this space. We’ll publish deep dives similar to our TLA+ write-up on logless reconfiguration, covering formal methods and other behind-the-scenes work on hard engineering problems, such as MongoDB 8.0 performance improvement challenges. Our vision is to carry the complexity so developers don’t have to—and to give them the agility & freedom to build the next generation of intelligent applications wherever they want.

At MongoDB my focus is on architectural improvements across the product stack. I've been impressed with the progression of MongoDB's capabilities and the team's continuous innovation ethos.

In this blog post, I’ll share some of my understanding of MongoDB’s history and how MongoDB became the de facto standard for document databases. I’ll also highlight select innovations we are actively exploring.

The dawn of NoSQL

During the "move fast and break things" era of Web 2.0, the digital landscape was exploding. Developers were building dynamic, data-rich applications at an unprecedented pace, and the rigid, tabular structures of legacy relational databases like Oracle and Microsoft SQL Server quickly became a bottleneck. A new approach was needed, one that prioritized developer productivity, flexibility, and massive scale. At the same time, JSON's popularity as a flexible, cross-language format for communicating between browsers and backends was surging. This collective shift toward flexibility gave rise to NoSQL databases, and MongoDB, with its native document-based approach, was at the forefront of the movement.

In the early days, there was a perception that MongoDB was great for use cases like social media feeds or product catalogs, but not for enterprise applications where data integrity is non-negotiable—like financial transactions. This view was never perfectly accurate, and it certainly isn't today. So, what created this perception? It came down to two main factors: categorization and maturity.

First, most early NoSQL databases were built on an “eventually consistent” model, prioritizing Availability and Partition Tolerance (AP) under the CAP theorem. MongoDB was an exception, designed to prioritize Consistency and Partition Tolerance (CP). But, in a market dominated by AP systems, MongoDB was often lumped in with the rest, leading to the imprecise label of having “light consistency.” Second, all new databases take time to mature for mission-critical workloads. Any established system-of-record database today has gone through many versions over many years to earn that trust. After more than 15 years of focused engineering, today MongoDB has the required codebase maturity, features, and proven track record for the most demanding enterprise applications.

The results speak for themselves. As our CEO Dev Ittycheria mentioned during the Q2 2026 earnings call, over 70% of the Fortune 100—as well as 7 of the 10 largest banks, 14 of the 15 largest healthcare companies, and 9 of the 10 largest manufacturers globally—are MongoDB customers. This widespread adoption by the world's most sophisticated organizations is a testament to a multi-year, deliberate engineering journey that has systematically addressed the core requirements of enterprise-grade systems.

MongoDB’s engineering journey: Building a foundation of trust

MongoDB’s evolution from being perceived as a niche database to an enterprise powerhouse wasn't an accident; it was the result of a relentless focus on addressing the core requirements of enterprise-grade systems. Improvements instrumental to this transformation include:

• High availability with replica sets: The first step was eliminating single points of failure. Replica sets were introduced as self-healing clusters that provide automatic failover, ensuring constant uptime and data redundancy. Later, the introduction of a Raft-style consensus protocol provided even more reliable and faster failover and leader elections, especially in the event of a network partition. This architecture is the foundation for MongoDB’s current multi-region or run-anywhere deployments, and even allows a single replica set to span multiple cloud providers for maximum resilience.

• Massive scalability with horizontal sharding: Introduced at the same time as replica sets, sharding is a native, foundational part of MongoDB. MongoDB built sharding to allow data to be partitioned across multiple servers, enabling virtually limitless horizontal scaling to support massive datasets and high-throughput operations. Advanced features like zone sharding further empower global applications by pinning data to specific geographic locations to reduce latency and comply with data residency laws like GDPR.
• Tunable consistency: Recognizing that not all data is created equal, MongoDB empowered developers with tunable read and write concerns. Within a single application, some data—like a 'page view count'—might not have the same consistency requirements as a 'order checkout value'. Instead of using separate, specialized databases for each use case, developers can use MongoDB for both. This moved the platform beyond a one-size-fits-all model, allowing teams to choose the precise level of consistency their application required per operation—from "fire and forget" for speed to fully acknowledged writes across a majority of replicas for guaranteed durability. This flexibility provides the best price/performance tradeoffs for modern applications.
• The game-changer, multi-document ACID transactions: From its inception, MongoDB has always provided atomic operations for single documents. The game-changing moment was the introduction of multi-document ACID transactions in 2018 with MongoDB 4.0, which was arguably the single most important development in its history. This feature, later extended to include sharded clusters, meant that complex operations involving multiple documents—like a financial transfer between two accounts—could be executed with the same atomicity, consistency, isolation, and durability (ACID) guarantees as a traditional relational database. This milestone shattered the biggest barrier to adoption for transactional applications. And the recently released MongoDB 8.2 is the most feature-rich and performant version of MongoDB yet.

• Strict security and compliance: To meet the stringent security demands of the enterprise, MongoDB layered in a suite of advanced security controls. Features like Role-Based Access Control (RBAC), detailed auditing, and Field-Level Encryption were just the beginning. The release of Queryable Encryption (to which we recently introduced support for prefix, suffix, and substring queries) marked a revolutionary breakthrough, allowing non-deterministic encrypted data to be queried without ever decrypting it on the server, ensuring data remains confidential even from the database administrator. To provide independent validation, MongoDB Atlas has achieved a number of internationally recognized security certifications and attestations, including ISO/IEC 27001, SOC 2 Type II, PCI DSS, and HIPAA compliance, demonstrating a commitment to meeting the rigorous standards of the world's most regulated industries.

The ultimate proof of enterprise readiness lies in real-world adoption. Today, MongoDB is trusted by leading organizations across the most demanding sectors to run their core business systems.

For example, Citizens Bank, one of the oldest and largest financial institutions in the United States, moved to modernize its fraud detection capabilities from a slow, batch-oriented legacy system. They built a new, comprehensive fraud management platform on MongoDB Atlas that allows for near real-time monitoring of transactions.

This use case in a highly regulated industry requires high availability, low latency, and strong consistency to analyze transactions in real-time and prevent financial loss—a direct refutation of the old "eventual consistency" criticism.

Another example is that of Bosch Digital, the software and systems house for the Bosch Group. Bosch Digital uses MongoDB for its IoT platform, Bosch IoT Insights, to manage and analyze massive volumes of data from connected devices—from power tools used in aircraft manufacturing, to sensors in vehicles. IoT data arrives at high speeds, in huge volumes, and in variable structures. This mission-critical use case demonstrates MongoDB's ability to handle the demands of industrial-scale IoT, providing the real-time analytics needed to ensure quality, prevent errors, and drive innovation.

Then there’s Coinbase, which relies on MongoDB to seamlessly handle the volatile and unpredictable cryptocurrency market. Specifically, Coinbase architected a MongoDB Atlas solution that would accelerate scaling for large clusters. The result was that Coinbase end-users gained a more seamless experience. Previously, traffic spikes could impact some parts of the Coinbase app. Now, users don’t even notice changes happening behind the scenes.

These are just a few examples; customers across all verticals, industries, and sizes depend on MongoDB for their most demanding production use cases. A common theme is that real-world data is messy, variable, and doesn't fit neatly into rigid, tabular structures.

The old adage says that if all you have is a hammer, everything looks like a nail. For decades, developers only had the relational "hammer." With MongoDB, they now have a modern tool that adapts to how developers work and the data they need to manage and process.

The road ahead: Continuous innovation

MongoDB is not resting on its laurels. The team is as excited about what the future holds as they were when MongoDB was first launched, and we continue to innovate aggressively to meet—and anticipate—the modern enterprise’s demands. Here are select improvements we are actively exploring.

A critical need we hear from customers is how to support elastic workloads in a price-performant way. To address this, over the past two years we’ve rolled out Search Nodes, which is a unique capability in MongoDB that allows scaling of search and vector workloads independent from the database to improve availability and price performance.

We are now working closely with our most sophisticated customers to explore how to deliver similar capabilities across more of MongoDB. Our vision is to enable customers to scale compute for high-throughput queries without over-provisioning storage, and vice versa. We can do all this while building upon what is already one of the strongest security postures of any cloud database, as we continue to raise the bar for durability, availability, and performance.

Another challenge facing large enterprises is the significant cost and risk associated with modernizing legacy applications. To solve this, we are making a major strategic investment in enterprise application modernization, and recently announced the MongoDB Application Modernization Platform. We have been engaged with several large enterprises in migrating their legacy relational database applications—code, data, and everything in between—over to MongoDB. This is not a traditional, manual migration effort capped by the number of bodies assigned. Instead, we are systematically developing Agentic tooling and AI-based frameworks, techniques, and processes that allow us to smartly migrate legacy applications into modern microservices-based architectures at scale.

One of the more exciting findings from a recent effort, working with a large enterprise in the insurance sector, was that optimized queries on MongoDB ran just as fast, and often significantly faster, than on their legacy relational database, even when schemas were translated 1:1 between relational tables and MongoDB collections, and lots of nested queries and joins were involved. Batch jobs implemented as complex stored procedures that took several hours to execute on the relational database could be completed in under five minutes, thanks to the parallelism MongoDB natively enables (for more, see the MongoDB Developer Blog).

Based on the incredible performance gains seen in these modernization projects, we're addressing another common need: ensuring fast queries even when data models aren't perfectly optimized. We are actively exploring improvements to our Query Optimizer that will improve lookup and join performance. While the document model will always be the most performant way to model your data, we are ensuring that even when you don't create the ideal denormalized data model, MongoDB will deliver performance that is at par or better than the alternatives.

Finally, developers today are often burdened with stitching together multiple services to build modern, AI-powered applications. To simplify this, the platform is expanding far beyond a traditional database, focused on providing a unified developer experience. This includes a richer ecosystem with integrated capabilities like Atlas Search for full-text search, Atlas Vector Search for AI-powered semantic search, and native Stream Processing to handle real-time data. We are already working on our first integrations, and continue to explore how embedding generation as a service within MongoDB Atlas, powered by our own Voyage AI models, can further simplify application development.

From niche to necessity

MongoDB began its journey as a (seemingly) niche NoSQL database with perceptions and tradeoffs that made it unsuitable for many core business applications. But, through a sustained and deliberate engineering effort, it has delivered the high availability, tunable consistency, ACID transactions, and robust security that enterprises demand. The perceptions of the past no longer match the reality of the present. When 7 of the 10 largest banks are already using MongoDB, isn’t it time to re-evaluate MongoDB for your most critical applications?