New Aggregation Pipeline Text Editor Debuts in MongoDB Compass

MongoDB
January 26, 2023
#Compass

There’s a reason why Compass is one of MongoDB’s most-loved developer tools: because it provides an approachable and powerful visual user interface for interacting with data on MongoDB. As part of this, Compass’s Aggregation Pipeline Builder abstracts away the finer points of MongoDB’s Query API syntax and provides a guided experience for developing complex queries.

But what about when you want less rather than more abstraction? That’s where our new Aggregation Pipeline Text Editor comes in.

Recently released on Compass, the Aggregation Pipeline Text Editor allows users to write free-form aggregations. While users could previously write and edit pipelines through a guided and structured builder organized by aggregation stage, a text-based builder can be preferable for some users. This new pipeline editor makes it easy for users to:

  • See the entire pipeline without having to excessively scroll through the UI

  • Stay “in the flow” when writing aggregations if they are already familiar with MongoDB’s Query API syntax

  • Copy and paste aggregations built elsewhere (like in MongoDB’s VS Code Extension) into Compass

  • Use built-in syntax formatting to make pipeline text “pretty” before copying it over from Compass to other tools

Screenshot of the Aggregation Pipeline Text Editor in Compass
The Aggregation Pipeline Text Editor in Compass. Notice how toward the top right you can click on “stages” to move back to the traditional stage-based Aggregation Pipeline Builder.

Ultimately, the addition of the Aggregation Pipeline Text Editor to Compass gives users more flexibility depending on how they want to build aggregations. For a more guided experience and to get result previews when adding each new stage, the existing Aggregation Pipeline Builder will work best for most users. But when writing free-form aggregations or copying and pasting aggregation text from other tools, the Aggregation Pipeline Text Editor may be preferable. It also previews the final pipeline output, rather than the stage-by-stage preview that exists today. Users will be able to access either both the traditional Aggregation Pipeline Builder and the new Pipeline Text Editor from directly within the Aggregations tab in Compass and can switch between the two views without losing their work.

To get access to the new Aggregation Pipeline Text Editor, make sure to download the latest version of Compass here. And as always, we welcome your continued feedback on how to improve Compass. If you have ideas for how to improve your experience with Compass you can submit them on our UserVoice platform here.

We’ll have even more great features coming in Compass soon. Keep checking back on our blog for the latest news!

← Previous

Modernizing Core Banking: A Shift Toward Composable Systems

Modernizing core banking systems with MongoDB can bring many benefits such as faster innovation, flexible deployment, and instant scalability. According to McKinsey & Company , it is critical for banks to modernize their core banking platforms with a “flexible back end” in order to stay competitive and adapt to new business models. With the emergence of better data infrastructure based on JSON and the ongoing evolution of software design, the next generation of composable core banking processes can be built on MongoDB's developer data platform, offering greater flexibility and adaptability than traditional systems. The current market: Potential core banking solutions Financial disruptors such as fintechs and challenger banks are growing their businesses and attracting customers by building on process-centric core banking systems, while traditional banks struggle with inflexible, legacy systems. As seen in Figure 1 below, two potential solutions are the core banking “platform” and “suite”. The platform solution involves using a single vendor and several closely integrated modules. It also includes a single, large database and a single roadmap. On the other hand, the suite solutions refers to using multiple vendors, multiple loosely integrated modules, multiple databases and roadmaps. However, both of these systems are inflexible and result in vendor lock-in, preventing the adoption of best-of-breed functionalities from other vendors. Figure 1: Core banking solutions: platform, suite and composable ecosystem. A new approach, known as a composable ecosystem as seen on the far right of Figure 1, is being adopted by some financial institutions. This approach consists of distinct independent services and functions, with the ability to incorporate "best of breed" functionality without major integration challenges, multiple loosely coupled roadmaps, and individual component deployment without vendor lock-in. This allows for specialization and the development of advanced individual components that can be combined to deliver the best products and services and is better at adopting new technologies and approaches. Composable ecosystems with MongoDB's developer data platform MongoDB’s developer data platform is the best choice for financial institutions to build a composable core banking ecosystem. Such an ecosystem is made up of four key building blocks as seen below in Figure 2: JSON, BIAN, MACH, and data domains. JSON is a widely-used data format in the financial industry, and MongoDB's BSON extension allows for the storage of additional data types. BIAN is a standard that defines a component business blueprint for banking, and MongoDB's technology supports BIAN and embodies MACH principles. MACH is a set of design principles for component-based architectures, and data domains enable the mapping of business capabilities to applications and data. By using MongoDB's developer data platform, financial institutions can implement flexible and scalable core banking systems that can adapt to the ever-changing market demands. Figure 2: MongoDB, the developer data platform for your core banking system. MongoDB in action: Core banking use cases Companies such as Temenos and Current have utilized MongoDB's capabilities to deliver innovative services and improve performance. As Tony Coleman, CTO of Temenos, said, "Implementing a good data model is a great start. Implementing a great database technology that uses the data model correctly, is vital. MongoDB is a really great fit for banking." MongoDB and Temenos have worked on a number of new, component-based services to enhance the Temenos product family. Financial institutions can embed Temenos components to deliver new functionality in their existing on-premises environments or through a full banking-as-a-service experience with Temenos T365, powered by MongoDB on various cloud platforms. Temenos has a cloud-first, microservices-based infrastructure built with MongoDB, which gives customers flexibility while improving performance. Current is a digital bank that was founded with the aim of providing its customers with a modern, convenient, and user-friendly banking experience. To achieve this, the company needed to build a robust, scalable, and flexible technology platform. Current decided to build its core technology ecosystem in-house, using MongoDB as the underlying database technology. "MongoDB gave us the flexibility to be agile with our data design and iterate quickly," said Trevor Marshall, CTO of Current. In addition, MongoDB's strong security features make it a secure choice for handling sensitive financial data. Overall, MongoDB's capabilities make it a powerful choice for driving innovation and simplifying landscapes in the financial sector. Conclusion In conclusion, the financial industry is in need of modernizing their core banking systems to stay competitive in the face of rising disruptors and new business models. A composable ecosystem, utilizing a developer data platform like MongoDB, offers greater flexibility and adaptability than traditional legacy systems. If you’d like to learn more about how MongoDB can optimize your core banking functionalities, take a look at our white paper: Componentized Core Banking: The next generation of composable banking processes built upon MongoDB .

January 26, 2023
Next →

Building AI With MongoDB: Integrating Vector Search And Cohere to Build Frontier Enterprise Apps

Cohere is the leading enterprise AI platform, building large language models (LLMs) which help businesses unlock the potential of their data. Operating at the frontier of AI, Cohere’s models provide a more intuitive way for users to retrieve, summarize, and generate complex information. Cohere offers both text generation and embedding models to its customers. Enterprises running mission-critical AI workloads select Cohere because its models offer the best performance-cost tradeoff and can be deployed in production at scale. Cohere’s platform is cloud-agnostic. Their models are accessible through their own API as well as popular cloud managed services, and can be deployed on a virtual private cloud (VPC) or even on-prem to meet companies where their data is, offering the highest levels of flexibility and control. Cohere’s leading Embed 3 and Rerank 3 models can be used with MongoDB Atlas Vector Search to convert MongoDB data to vectors and build a state-of-the-art semantic search system. Search results also can be passed to Cohere’s Command R family of models for retrieval augmented generation (RAG) with citations. Check out our AI resource page to learn more about building AI-powered apps with MongoDB. A new approach to vector embeddings It is in the realm of embedding where Cohere has made a host of recent advances. Described as “AI for language understanding,” Embed is Cohere’s leading text representation language model. Cohere offers both English and multilingual embedding models, and gives users the ability to specify the type of data they are computing an embedding for (e.g., search document, search query). The result is embeddings that improve the accuracy of search results for traditional enterprise search or retrieval-augmented generation. One challenge developers faced using Embed was that documents had to be passed one by one to the model endpoint, limiting throughput when dealing with larger data sets. To address that challenge and improve developer experience, Cohere has recently announced its new Embed Jobs endpoint . Now entire data sets can be passed in one operation to the model, and embedded outputs can be more easily ingested back into your storage systems. Additionally, with only a few lines of code, Rerank 3 can be added at the final stage of search systems to improve accuracy. It also works across 100+ languages and offers uniquely high accuracy on complex data such as JSON, code, and tabular structure. This is particularly useful for developers who rely on legacy dense retrieval systems. Demonstrating how developers can exploit this new endpoint, we have published the How to use Cohere embeddings and rerank modules with MongoDB Atlas tutorial . Readers will learn how to store, index, and search the embeddings from Cohere. They will also learn how to use the Cohere Rerank model to provide a powerful semantic boost to the quality of keyword and vector search results. Figure 1: Illustrating the embedding generation and search workflow shown in the tutorial Why MongoDB Atlas and Cohere? MongoDB Atlas provides a proven OLTP database handling high read and write throughput backed by transactional guarantees. Pairing these capabilities with Cohere’s batch embeddings is massively valuable to developers building sophisticated gen AI apps. Developers can be confident that Atlas Vector Search will handle high scale vector ingestion, making embeddings immediately available for accurate and reliable semantic search and RAG. Increasing the speed of experimentation, developers and data scientists can configure separate vector search indexes side by side to compare the performance of different parameters used in the creation of vector embeddings. In addition to batch embeddings, Atlas Triggers can also be used to embed new or updated source content in real time, as illustrated in the Cohere workflow shown in Figure 2. Figure 2: MongoDB Atlas Vector Search supports Cohere’s batch and real time workflows. (Image courtesy of Cohere) Supporting both batch and real-time embeddings from Cohere makes MongoDB Atlas well suited to highly dynamic gen AI-powered apps that need to be grounded in live, operational data. Developers can use MongoDB’s expressive query API to pre-filter query predicates against metadata, making it much faster to access and retrieve the more relevant vector embeddings. The unification and synchronization of source application data, metadata, and vector embeddings in a single platform, accessed by a single API, makes building gen AI apps faster, with lower cost and complexity. Those apps can be layered on top of the secure, resilient, and mature MongoDB Atlas developer data platform that is used today by over 45,000 customers spanning startups to enterprises and governments handling mission-critical workloads. What's next? To start your journey into gen AI and Atlas Vector Search, review our 10-minute Learning Byte . In the video, you’ll learn about use cases, benefits, and how to get started using Atlas Vector Search.

April 25, 2024