MongoDB Updates
The newest releases and freshest updates
Recap of Product Announcements at MongoDB.local London, 2023
September 26, 2023
Updates
New Intelligent Developer Experiences for Compass, Atlas Charts, Relational Migrator, and Docs
This post is also available in: Deutsch , Français , Español , Português . Today, MongoDB announced a range of innovations in its developer data platform, creating new, intelligent developer experiences in familiar tools like MongoDB Compass, Atlas Charts, Relational Migrator, and MongoDB Documentation that radically simplify and accelerate how developers build modern applications. These new experiences provide developers with guided and intelligent assistance for their development processes in: MongoDB Compass: Where developers can use natural language to compose everything from simple queries to sophisticated, multi-stage aggregations. MongoDB Relational Migrator: Where developers can convert SQL queries to MongoDB Query API syntax. MongoDB Atlas Charts: Where developers can use natural language to generate basic data visualizations. MongoDB Documentation: Where developers can ask questions to an intelligent chatbot, built on top of MongoDB Atlas and Atlas Vector Search, to enable lightning-fast information discovery and troubleshooting during software development. Developer time is one of the most precious commodities in any organization, and with business and customer expectations continuing to rise, developers are under increasing pressure to deliver applications quickly. With more intelligent experiences across the MongoDB developer data platform, it is now simpler and easier than ever to build modern applications for virtually any use case. Natural Language Queries in Compass Building queries and aggregations is one of the most prominent developer use cases for Compass , MongoDB’s popular, downloadable GUI tool. Compass’ new, intelligent experience allows developers to use natural language to compose sophisticated aggregations to query, transform, and enrich data, reducing the complexity and learning curve to build queries into application code. The new experience is being released in Public Preview in version 1.40.0 and will be rolled out incrementally to users starting today until the end of October. To get started, make sure you have 1.40.0 downloaded on your machine and have access to the feature. Then you can navigate to the Documents tab and click on the Generate Query button in the query bar, which opens a second bar below the standard query bar where you can enter natural language prompts to generate the Query API syntax for you to execute against your data. Be sure to hit the “thumb’s up” or “thumb’s down” button to rate the helpfulness of the query generated. SQL Query Conversion in Relational Migrator Migrations are part of many developers’ journeys with MongoDB. Earlier this summer at MongoDB.Local NYC, we announced Relational Migrator to help teams with these projects, and we’re continuing to make it easier to modernize application code. Many legacy systems have hundreds, if not thousands of SQL queries that must be modernized as part of any migration effort, and that can be a time-consuming, if not daunting task. Now in Private Preview, developers can use Relational Migrator to convert existing SQL queries and stored procedures into development-ready MongoDB Query API syntax. With SQL query conversion, developers can leverage Relational Migrator to eliminate the manual effort of creating MongoDB queries at scale - speeding up migration projects. SQL query conversion is currently available in Private Preview, and access can be requested directly from the latest version of Relational Migrator. Natural Language Support in Atlas Charts Atlas Charts is the best way for developers to visualize Atlas data. By offering an effortless and powerful solution for gaining data-driven insights, Charts empowers developers and the businesses they help scale. What has always been easy is now becoming more intelligent too! Available in Private Preview, a new natural language mode allows developers to visualize their data through a simple language query, for example: “show me a comparison of annual revenue by country and product.” This is just the start. Later this year, natural language support will extend to more complex queries and chart types. Sign up today to try out natural language support for building charts! Stay tuned for more updates from the team and check out our documentation to learn more about what’s supported by natural language during Private Preview! Intelligent Chatbot for MongoDB Documentation Documentation is critical to the developer experience, making it easier to discover product features and capabilities and troubleshoot common challenges during software development. MongoDB is now super-charging your experience with an intelligent chatbot that improves information discovery by surfacing and summarizing the most relevant documentation. Built with MongoDB Atlas and Atlas Vector Search, the chatbot allows you to ask questions in natural language like “How do I get started with MongoDB Atlas?” or “How do I add a new IP address to the IP access list for my Atlas project?” and receive a response with reference articles, code examples, and other relevant information. MongoDB will also be open-sourcing and providing educational materials about how we built the intelligent chatbot, making it that much easier for others in the community to use the power of MongoDB Atlas and Atlas Vector Search to create dynamic and educational experiences for their end users. Data Privacy and Security MongoDB is trusted by some of the world's most security-conscious organizations, who use the developer data platform’s robust data security and privacy controls to manage their most sensitive data assets. To maintain this trust, these new developer experiences will always be transparent about what data is accessed and used, allowing customers to make informed decisions within the boundaries of their unique security, privacy, and compliance concerns. Get Started Today With new, intelligent features that allow developers to interact with their data using natural language in Compass, Relational Migrator, and Charts, as well as an intelligent chatbot for MongoDB Documentation, it’s easier than ever to take advantage of the flexibility and scalability of MongoDB's document data model to build any class of application. If you have feedback on these experiences, you can enter a suggestion in our user feedback portal .
Introducing a Local Experience for Atlas, Atlas Search, and Atlas Vector Search with the Atlas CLI
This post is also available in: Deutsch , Français , Español , Português . Today, MongoDB is pleased to announce in Public Preview a new set of features for building software locally with MongoDB Atlas, giving developers greater flexibility and reducing operational overhead throughout the entire software development lifecycle. Developers can now develop locally with MongoDB Atlas deployments, including Atlas Search and Vector Search , using the Atlas CLI , empowering them to create full-text search or AI-powered applications no matter their preferred environment for building with MongoDB. Developers can use the Atlas CLI to set up, connect to, and automate common management tasks from early development through testing, staging, and production. For full-text search use cases, developers can now use the Atlas CLI to create and manage Atlas Search indexes regardless of whether they are working locally or in the cloud. Similarly, developers building applications powered by semantic search and generative AI on MongoDB can now use the Atlas CLI to create and manage local development instances with Vector Search indexes regardless of their development environment. Developer time is one of the most precious commodities in any organization building innovative new application experiences. But all too frequently, developers are burdened with managing repeatable tasks such as setting up development environments. They also often have to wrestle with the cognitive overhead of switching between different user experiences for local versus cloud development, distracting from delivering value. By giving developers the power of Atlas at their fingertips no matter their preferred development environment, MongoDB continues to expand the scope and capabilities of its developer data platform while placing a premium on developer experience. Create a Local Atlas Database Ready to create a local Atlas database, but don’t have the Atlas CLI yet? It’s easy to install with your favorite package manager. To install the Atlas CLI with Homebrew, use the following command: brew install mongodb-atlas In addition to installing via the Homebrew package manager, you can install the MongoDB Atlas CLI via Apt, Yum, Chocolatey, directly downloading the binary, or pulling the Docker image (learn more about our documentation ). You can also download it directly from the MongoDB Download Center . To create a local Atlas deployment with default settings in interactive mode, enter: atlas deployments setup --type local If you want to list your Atlas deployments enter: atlas deployments list If you’re authenticated to Atlas, you will see both your local and cloud Atlas deployments. If you aren’t authenticated to Atlas, you will only see your local deployments. Get Started with Local Atlas Search Building an application with a full-text search feature powered by Atlas Search? If you’re a developer who tends to build and prototype locally, you may be interested in using the Atlas CLI to work with Atlas Search in your local environment. To get started, first, connect to the local deployment on which you’d like to create a Search index: atlas deployments connect Next, you can use the MongoDB Shell to create your Search index. Below you’ll see an example of how to create an Atlas Search index: db.YOURCOLLECTION.createSearchIndex( "example-index", { mappings: { dynamic: true } } ) Then, if you want to run a query you can use the $search stage of an aggregation pipeline. You can learn more about managing Atlas Search indexes in our documentation . Get Started with Local Vector Search If you’re building an application with generative AI or semantic search and MongoDB Atlas, chances are you’ll be interested in our Atlas Vector Search offering. And now with the Atlas CLI, you can work with Vector Search in the cloud and your local environment. To get started with Vector Search locally you can use MongoDB Shell to create a Vector Search index. Notice that this is similar to the Atlas Search example above, except that in this case there is a vector embedding accounted for in search index creation. db.YOURCOLLECTION.createSearchIndex({ "mappings": { "dynamic": true, "fields": { "plot_embedding": { "type": "knnVector", "dimensions": 1536, "similarity": "euclidean" } } } } ) To learn more about running Vector Search queries visit our documentation . Additionally, if you're already familiar with handling your cloud Search indexes using the Atlas CLI, you'll appreciate a fresh set of interactive commands designed to help you efficiently manage Atlas Search and Vector Search indexes both locally and in the cloud: atlas deployments search indexes create From there you can move through an interactive flow that guides you through index creation. For detailed instructions visit our tutorial . Ready to Move to the Cloud? If you’re ready to create an Atlas database in the cloud, that is easy to do with the Atlas CLI. Simply use the following command: atlas deployments setup --type atlas From there, the setup wizard will guide you to: Register for an Atlas account or authenticate to an existing account Create a free MongoDB Atlas database Load sample data Add your IP address to the access list Create a database user and password Connect to the cluster using the MongoDB Shell ( mongosh ) so you can begin interacting with your data To learn more about the Atlas CLI, visit our documentation . And be sure to let us know what you think of the Atlas CLI in our user feedback portal . With the new local experience with the Atlas CLI, it’s easier than ever to work with your data on Atlas no matter your preferred development environment. Get started today with the Atlas CLI as the ultimate developer tool to manage MongoDB Atlas, including Atlas Search and Vector Search, throughout the entire software development lifecycle, from your local environment all the way to the cloud.
Introducing Atlas for the Edge
This post is also available in: Deutsch , Français , Español , Português . We are thrilled to introduce MongoDB Atlas for the Edge at MongoDB.local London. This new solution is designed to streamline the management of data generated across various sources at the edge, including devices, on-premises data centers, and the cloud. Edge computing, which brings data processing closer to end-users, offers significant advantages. At the same time, it often proves challenging due to complex networking, data volume management, and security concerns, which can deter many organizations. They are also costly to build, maintain, and scale. Some challenges organizations face include: Significant technical expertise to manage the complexity of networking and high volumes of distributed data required to deliver reliable applications that run anywhere Stitching together hardware and software solutions from multiple vendors, resulting in complex and fragile systems that are often built using legacy technology that is limited by one-way data movement and requires specialized skills to manage and operate Constant optimization of edge devices due to their constraints — like limited data storage and intermittent network access — which makes keeping operational data in sync between edge locations and the cloud difficult Security vulnerabilities and frequent firmware patches and updates to ensure data privacy and compliance MongoDB Atlas for the Edge simplifies all of these manual tasks. It allows MongoDB to run on diverse edge infrastructure, from self-managed, on-premises servers to cloud deployments offered by major cloud providers. Data seamlessly flows between and is kept synchronized across all sources, ensuring real-time data delivery with minimal latency. With MongoDB Atlas for the Edge, organizations can now use a single, unified interface to deliver a consistent and frictionless development experience from the edge to the cloud — and everything in between. Together, the capabilities included with MongoDB Atlas for the Edge allow organizations to significantly reduce the complexity of building edge applications and architectures: Run MongoDB on a variety of edge infrastructure for high reliability with ultra-low latency: With MongoDB Atlas for the Edge, organizations can run applications on MongoDB using a wide variety of infrastructure, including self-managed, on-premises servers, such as those in remote warehouses or hospitals, in addition to edge infrastructure managed by major cloud providers including AWS, Google Cloud, and Microsoft Azure. For example, data stored in MongoDB Enterprise Advanced on self-managed servers can be automatically synced with MongoDB Atlas Edge Server on AWS Local Zones and MongoDB Atlas in the cloud to deliver real-time application experiences to edge devices with high reliability and single-digit millisecond latency. MongoDB Atlas for the Edge allows organizations to deploy applications anywhere, even in remote, traditionally disconnected locations — and keep data synchronized between edge devices, edge infrastructure, and the cloud — to enable data-rich, fault-tolerant, real-time application experiences. Atlas Edge Server is now in private preview, learn more on our product page . Run applications in locations with intermittent network connectivity: With Atlas Edge Server and Atlas Device Sync , organizations can use a pre-built, local-first data synchronization layer for applications running on kiosks or on mobile and IoT devices to prevent data loss and improve offline application experiences. MongoDB Edge Servers can be deployed in remote locations to allow devices to sync directly with each other—without the need for connectivity to the cloud—using built-in network management capabilities. Once network connectivity is available, data is automatically synchronized between devices and the cloud to ensure applications are up to date for use cases like inventory and package tracking across supply chains, optimizing delivery routes in remote locations, and accessing electronic health records with intermittent network connectivity. Build and deploy AI-powered applications at the edge: MongoDB Atlas for the Edge provides integrations with generative AI and machine learning technologies to provide low-latency, intelligent functionality at the edge directly on devices—even when network connectivity is unavailable. For example, MongoDB Atlas Search and Atlas Vector Search make it faster and easier to build intelligent applications with search and generative AI capabilities that take advantage of large language model vector embeddings (numeric representations of data such as text, images, and audio). Once embeddings are generated and stored in MongoDB Atlas, edge applications running on the Atlas Device SDK (formerly Realm) —a fast, scalable platform with mobile-to-cloud data synchronization that makes building real-time, reactive mobile applications easy—can use embeddings stored locally for use cases like real-time image similarity search and classification to identify potential product defects on factory lines. Developers can also use the Atlas Device SDKs to build, train, deploy, and manage machine learning models on edge devices using popular frameworks like CoreML, TensorFlow, and PyTorch for customized applications that take advantage of real-time data. Store and process real-time and batch data from IoT devices to make it actionable: With MongoDB Atlas Stream Processing , organizations can ingest and process high-velocity, high-volume data from millions of IoT devices (e.g., equipment sensors, factory machinery, medical devices) in real-time streams or in batches when network connectivity is available. Data can then be easily aggregated, stored, and analyzed using MongoDB Time Series collections for use cases like predictive maintenance and anomaly detection with real-time reporting and alerting capabilities. MongoDB Atlas for the Edge provides all of the tools necessary to process and synchronize virtually any type of data across edge locations and the cloud to ensure consistency and availability. Easily secure edge applications for data privacy and compliance: MongoDB Atlas for the Edge helps organizations ensure their edge deployments are secure with built-in security capabilities. The Atlas Device SDK provides out-of-the-box data encryption at rest, on devices, and in transit over networks to ensure data is protected and secure. Additionally, Atlas Device Sync provides fine-grained role-based access, with built-in identity and access management (IAM) capabilities that can also be combined with third-party IAM services to easily integrate edge deployments with existing security and compliance solutions. Some of the leading organizations are leveraging Atlas for the Edge today. For example: Cathay Pacific , Hong Kong’s home airline providing passenger and cargo services to destinations around the world, understood the need for digital transformation in their critical pilot briefing process and in-flight operations. With MongoDB Atlas, they were the very first to digitize their flight operations process with an iPad app, Flight Folder, enabling one of the first zero paper flights in the world in September of 2019. MongoDB’s developer data platform met their requirements for this and many other projects, successfully improving costs, operational efficiency, and accuracy, while also reducing environmental impact. Read the case study to learn more. Cloneable provides low/no-code tools to enable instant deployment of AI applications to a spectrum of devices—mobile, IoT devices, robots, and beyond. “We collaborated with MongoDB because Atlas for the Edge provided capabilities that allowed us to move faster while providing enterprise-grade experiences,” said Tyler Collins, CTO at Cloneable. “For example, the local data persistence and built-in cloud synchronization provided by Atlas Device Sync enables real-time updates and high reliability, which is key for Cloneable clients bringing complex, deep tech capabilities to the edge. Machine learning models distributed down to devices can provide low-latency inference, computer vision, and augmented reality. Atlas Vector Search enables vector embeddings from images and data collected from various devices to allow for improved search and analyses. MongoDB supports our ability to streamline and simplify heavy data processes for the enterprise.” To learn more about the solution announced today, and find out how retailers and healthcare organizations are leveraging the solution, please visit the web page for Atlas for the Edge .
View and Analyze Your Monthly MongoDB Atlas Usage with Cost Explorer
In today's macroeconomic climate, knowing where your money's going is a big deal. From optimizing costs to boosting efficiency, understanding your software expenses can be a total game-changer for your business. That’s why we’re excited to announce the release of Cost Explorer in MongoDB Atlas. Cost Explorer is a new visual interface available in the Billing section of the Atlas UI that is meant to help you view and analyze your monthly MongoDB Atlas usage in one convenient location. How can Cost Explorer help you? Cost Explorer allows you to easily filter your Atlas usage data by what’s most important to you and your business, with filters to segment your view by organization (if you have cross-org billing enabled), projects, clusters, or services, within a time window of up to 18 months. With Cost Explorer, you can now quickly pinpoint trends or outliers in your month-over-month usage to identify opportunities to potentially improve or optimize your Atlas usage going forward. If you’re looking for additional customization beyond what is available in Cost Explorer, you can also create your own billing dashboards in Atlas Charts that are fully tailored to your needs. Cost Explorer is viewable for any Atlas user assigned the Organization Owner, Billing Admin, or Organization Billing Viewer roles. To learn more about Cost Explore and how to manage your Atlas billing, view our documentation on managing billing .
Resource Tags in Billing Invoices Now Generally Available
In June, we announced MongoDB’s Atlas new resource tagging capability - built to provide a simple and easy way to organize and manage database deployments at scale. Today, we are pleased to announce the availability of these tags in billing invoice CSVs and Admin API operations. Leveraging tags within billing invoices empowers customers to categorize spending based on custom dimensions (such as cost center) and streamline financial reporting. Why we built Resource Tags in Billing Invoices Today, billing-related tags are the recommended way to allocate costs based on dimensions unique to customer organizations. Previously, customers were exporting CSV invoices or using the Admin API to manually append additional metadata, and generate custom financial reports resulting in a significant amount of time lost. Other customers were naming their deployments after internal billing references instead of meaningful names for their development teams. Addressing this leads to an enhanced user experience, streamlined reporting processes, and ensures invoices can meet the unique cost allocation needs of our customers. How customers can benefit from Resource Tags in Billing Invoices In today's cost-conscious micro-environment, allowing customers to segment their spending based on custom metadata enables them to optimize financial reporting by categorizing expenses based on custom dimensions that go beyond project or cluster names. With the flexibility to break down costs according to specific business units, engineering environments, or individual lines of business, organizations of all sizes can tailor their cost allocation requirements accordingly. By streamlining the reporting process and providing comprehensive financial visibility, customers are empowered with more informed decisions. How to get started Customers can get started today by adding tags to new or existing deployments using the Atlas UI, Admin API, or CLI. Once added, the tag keys will appear as column headings in the billing invoice CSV export and as a field in the Invoices payload for the Admin API calls. Tags will typically appear on Invoice line items within 48 hours of being added to the deployment. Also, tags will not appear on cluster related line items that are not associated with a tag and line items not billed at the cluster level are not eligible for tagging at this time. The example below shows an example CSV export for the fictitious project, Leafy. This organization has two tag keys (Environment and Cost-Center) associated with different clusters. Notice that the cluster named “Blue” does not have the Cost-Center tag applied, so those cells are blank. Resource tagging best practices Tagging approaches can range from simple to complex, depending on the needs of the organization. We have outlined tips and best practices to get started: Do not include any sensitive information such as Personally Identifiable Information (PII) or Protected Health Information (PHI) in resource tag keys or values. Use a standard naming convention - including spelling, case, and punctuation for all tags. Define and communicate a strategy that enforces mandatory tags on all database deployments. We recommend you start with identifying the environment and the application/service/workload. Use namespaces or prefixes to easily identify tags owned by different business units. Use automated tools such as Terraform or the Admin API to programmatically manage database deployments and tags. Implement a tag governance routine that regularly checks for untagged or improperly tagged deployments. Conclusion Tags applied to Billing Invoices meet customers' critical needs by allowing them to categorize spending based on custom dimensions, optimizing financial reporting, and increasing trust and efficiency. Customers gain the flexibility to break down costs according to various business units, engineering environments, or specific lines of business, making financial analysis more efficient and more insightful than ever before. Sign up for MongoDB Atlas , our cloud database service, to see tagging in action, and for more information, see Atlas Resource Tagging .
Boosting Performance and Insights with MongoDB Atlas and New Relic
In order to keep up with the demands of the modern business landscape, organizations must prioritize monitoring and optimizing application performance. Today, we're excited to announce a powerful collaboration between MongoDB Atlas, the leading database-as-a-service platform, and New Relic , the renowned monitoring and observability solution. This integration enables users to seamlessly monitor, analyze, and optimize their MongoDB deployments with efficiency and ease. Observability to ensure availability and uptime Monitoring a MongoDB Atlas deployment is now simpler than ever with the integration of New Relic. By connecting the two platforms, users can effortlessly monitor the performance of their MongoDB metrics, such as latency, throughput, and error rates, directly within the New Relic user interface. With real-time visibility into the health and performance of their databases, developers, and operations teams can quickly identify potential bottlenecks and proactively address issues before they impact end-users. Similarly, administrators can get an immediate high-level view of the health and availability of their MongoDB databases. Intelligent alerting and notifications The combination of MongoDB Atlas and New Relic empowers users to set up intelligent alerts and notifications tailored to their specific business requirements. Leveraging New Relic's alerting capabilities, users can create custom alert policies based on performance metrics and query patterns. Whether it's a sudden increase in response times or an unexpected spike in query access patterns, teams can receive timely notifications via email, Slack, or other preferred channels, enabling them to take immediate action. Powerful dashboarding and reporting Users can also take advantage of comprehensive dashboarding and reporting capabilities. With customizable dashboards and rich visualizations, users can gain real-time insights into the performance of their MongoDB clusters. Additionally, New Relic's reporting tools enable teams to generate detailed reports on database performance, query analytics, and overall system health, empowering them to make data-driven decisions and track improvements over time. By combining the strengths of these two powerful platforms, users can now unlock a new level of control and efficiency in managing their MongoDB databases. From streamlined monitoring and analysis to improved troubleshooting and enhanced collaboration, this integration leads organizations to proactively optimize their applications, ensure scalability, and deliver exceptional user experiences. With MongoDB Atlas and New Relic working cohesively, businesses can stay ahead in today's rapidly evolving digital landscape, where performance and efficiency are key differentiators. If you’d like to see the MongoDB Atlas and New Relic integration in action, sign up for MongoDB Atlas , our cloud database service, and learn more about New Relic’s monitoring and observability capabilities .
Improving the Node.js Driver’s SCRAM-SHA-256 Support
MongoDB always strives to offer best-in-class features, functionality, and security. A number of authentication mechanisms currently exist to verify the identity of a connecting client to your cluster, and when using the Salted Challenge Response Authentication Mechanism (SCRAM) there are two possible hashing functions: SCRAM-SHA-1 and SCRAM-SHA-256 . The MongoDB Driver Authentication Specification outlines that when attempting to authenticate using SCRAM: “If SCRAM-SHA-256 is present in the list of mechanism, then it MUST be used as the default; otherwise, SCRAM-SHA-1 MUST be used as the default [...]”. A MongoDB Server ( mongos or mongod ) can be configured with a list of possible authenticationMechanisms . As a result, MongoDB can be configured to return new authentication mechanisms which can upgrade already running applications to more secure authentication. This is the case when SCRAM-SHA-256 is added to a cluster that previously only supported SCRAM-SHA-1 . Prior to hashing passwords with SHA-256, they will first be prepared using SASLprep . The MongoDB Node.js driver leverages an external library ( saslprep ) for this functionality, which was an optional dependency and only used if available. Though a number of checks were in place to ensure the library was available (and loaded), an edge case was found where these checks could fail and report availability incorrectly. Potential use Most applications won’t experience this issue, however if your Node.js project is being bundled using an alternate bundler (such as webpack ) it’s possible a variation of this issue may surface. If your application was affected, it would be unable to connect to your MongoDB cluster. The stack trace from the error that would be thrown should include a call to continueScramConversation similar to the following examples: { "errorType": "TypeError", "errorMessage": "saslprep is not a function", "stack": [ "TypeError: saslprep is not a function", " at continueScramConversation ([...]/index.js:xxx:yyy)", [...] ] } TypeError: l is not a function at continueScramConversation (/app/webpack:[...]/mongodb/lib/core/auth/scram.js:xxx:yy) Note that Mongoose applications can also be affected, as Mongoose wraps the Node.js driver: TypeError: (0 , o.saslprep) is not a function at continueScramConversation (/app/webpack:[...]/mongoose/node_modules/mongodb/lib/cmap/auth/scram.js:xxx:yy) Next steps The underlying issue was addressed in versions 5.7.0 , 4.17.0 and 3.7.4 of the MongoDB Node.js driver, so depending on the version of the driver being used by your application a minor version update will address this. Upgrading your application’s libraries and deploying to production may not always be possible in a timely fashion. If this is the case and you happen to hit the issue described above a workaround would be to append the authMechanism option to your connection string with a value of SCRAM-SHA-1 as follows: mongodb+srv://xyz.mongodb.net/test?authMechanism=SCRAM-SHA-1 This will force the driver to attempt authorization using the SCRAM-SHA-1 hashing algorithm. Note that connection string changes would still require the application to be restarted for those changes to take effect.
Crafting Precision: Query Enhancements in MongoDB 7.0
Writing performant queries is the holy grail for any developer working with databases. It’s akin to an art form – balancing speed, efficiency, and functionality in one seamless package. At MongoDB .local Chicago, we announced the General Availability of MongoDB 7.0 , the latest version of our database which includes a number of new features targeted at making the application development experience streamlined and boosting developer productivity. Constructing efficient queries frequently entails navigating complex optimization and index management processes. With MongoDB 7.0, this process has been significantly streamlined. Introducing compound wildcard indexes Now, imagine writing less code and utilizing fewer system resources to get the results you need faster. With MongoDB 7.0, Compound Wildcard Indexes make this dream a reality. Compound wildcard indexes support creating compound indexes on a combination of any field and all sub-fields in an embedded sub-document. This supports performant queries against a combination of an unknown or arbitrary field and an always-present field. For example, consider a collection named customer_attributes with a diverse schema. Not every customer has every attribute filled out. Compound wildcard indexes can now support queries on Region or Tier and any of the sub-fields under Attributes , thus embracing MongoDB’s schema flexibility. db.customer_attributes.createIndex({"Region": 1, "Attributes.$**": 1}); This index could support a query like this, efficiently and easily: db.customer_attributes.find({"Region": "North America", "Attributes.CSM": "John Doe"}); Bitwise operators: The power in your pipeline Before MongoDB 7.0, developers can leverage faster computations with Bitwise Operators in the Aggregation Pipeline . While this has been available for some time with find and update queries, MongoDB 7.0 allows you to compute common bitwise operators within aggregations, right on the database, significantly enhancing query performance, especially for application-driven and operational analytics. Let's take an example of filtering documents based on binary data flags: { $match: { $expr: { $eq: [{$bitAnd: ["$userFlags", 4]}, 4] } } } This filter checks whether the fourth bit of the userFlags field is set, a common scenario in permission systems. Percentile operators: Unlocking efficient data analysis Calculating percentiles is a cornerstone of data analysis. With MongoDB 7.0, Approximate Percentile Operator enables developers to perform these calculations directly on the database server, significantly enhancing efficiency and reducing complexity. For example, to find the 90th percentile of response times in a weblogs collection: { $group: { _id: null, responseTime90thPercentile: { $percentile: { input: "$responseTime", p: [0.9], method: 'approximate'} } } } Role-based data access with $$USER_ROLES The new $$USER_ROLES variable introduces an elegant way to deliver user-specific views right from your queries, offering an enhanced layer of security and data privacy. No more separate views for each user role – now it’s seamless and simplified. Consider a sales application, where account data can be seen by everyone but sensitive information like contract details can only be seen by specific managers. By leveraging the new variable and setting up a view as below, you can ensure only managers with that specific role can view the data: pipeline = [ { $set: { "contractDetails": { $cond: { if: { $in: [ "csmApp", '$$USER_ROLES.role' ] }, then: "$contractDetails", else: "$$REMOVE" } } } } ] db.createView('accounts_view', 'accounts', pipeline ) This ensures that users can only view the data tied to their account, based on their role privileges. Conclusion: Elevating the developer experience MongoDB 7.0’s new query features are more than just a set of tools; they represent a profound upgrade in the developer experience. With these features, you're not just writing queries - you are sculpting your application’s data interaction in the most efficient way, right from the start. From Compound Wildcard Indexes to new operators in the aggregation pipeline, MongoDB 7.0 empowers developers to write cleaner, faster, and more intuitive queries. With less time spent on query optimization, you can focus on what truly matters: building amazing applications. Build your next application with a streamlined development experience .
What's New in Atlas Charts: Summer 2023 Release Roundup
Today, we’re excited to announce a series of updates to Atlas Charts . The Charts team is constantly adding enhancements to make visualizing data from MongoDB Atlas collections quick, easy, and powerful. With this summer’s release, we are introducing: Expanded chart customization options in the Charts Embedding SDK An out-of-the-box billing dashboard for easily monitoring your Atlas billing data Support for a new chart type with candlestick charts, and Query execution improvements for more reliable visualization of even your largest datasets. Let’s go through these updates one by one. Expanded charts customization options in the Charts embedding SDK Embedded charts and dashboards can be a useful way to share information across an organization and even with your customers. Over the last year, we have made many improvements to getting started with and implementing embedded charts and dashboards. With this release, we focused on the look and feel of your embedded charts. The Charts Embedding SDK now allows for greater customization across attributes like chart titles, descriptions, axes options, channel options, chart color palette, and conditional formatting options. Providing access to these chart specifications matches many of the customization options that developers rely on when building charts inside Atlas. An out-of-the-box Atlas billing dashboard Next, let’s talk about discovering insights from your Atlas billing data. Over the past two years, MongoDB has made visualizing billing data in Atlas Charts possible . Implementing this dashboard required quite a bit of effort in the past, but we heard from many teams that this dashboard is essential to the monitoring of their Atlas bill. So, we worked to build a more streamlined process for enabling these insights. Enter the new billing dashboard. The new billing dashboard in Atlas Charts is as simple as a button click on our new Ingestions page. Simply click Add and open and you’ll see a billing dashboard ready for exploration. The new billing dashboard allows for customization, adding dashboard filters, and getting new views into your data with the ability to add other charts based on your interests. Support for a new chart type: Candlestick charts The third update in this release is support for candlestick charts. While many developers leverage the wide variety of chart types available in Charts, we are always paying attention to additional chart types that can enable even more use cases for visualizing your Atlas data. This is why we have added candlestick charts with this latest release. Candlestick charts are commonly used for financial data, richly representing financial trading trends and price fluctuations. Query execution improvements for more reliable visualization of even your largest datasets Finally, let’s talk about query execution in Charts. One of the key characteristics of Charts is that it works natively with your Atlas data, requiring no data movement or duplication. Developers can simply select a data source to explore and immediately discover insights about application data. However, that doesn’t mean things are simple on the backend when creating a chart. Charts does a lot of work to efficiently render your visualizations as seamlessly as possible. And depending on your cluster configuration and the complexity of the chart you are creating (and the query driving it), Charts can take some time to render and produce the information you are aiming to visualize. Historically, Charts had a timeout duration of 120 seconds. Queries taking longer than this, would fail and not render the desired visualization. With this update, Charts users will see: Improved rendering of more complex queries as we have extended the timeout duration to 10 minutes, and More helpful in-product notification that lets you know when a query may take additional time to run, so you can continue either creating other visualizations or doing other work, until your query finishes. If you have queries requiring a longer timeout duration than 10 minutes, you can contact the Charts team through support. Note: Query Execution will initially be available through contacting Customer Support, or your existing Customer Success team, to opt-in. The feature will become Generally Available to all accounts in October, 2023. With that, we hope these enhancements make Charts even easier and more powerful to use. Over the coming weeks, we’ll go a level deeper, diving into some of these features with more detail. In the meantime, go build some visualizations in Charts to get hands-on and see for yourself! New to Atlas Charts? Get started today by logging into or signing up for MongoDB Atlas , deploying or selecting a cluster, and activating Charts for free.
MongoDB Announces Queryable Encryption with Equality Query Type Support
The general availability of Queryable Encryption offers end-to-end encryption of sensitive data while preserving the ability to run equality queries on that encrypted data, helping customers meet the strictest data privacy requirements. This technology allows developers to query encrypted sensitive data in a simple, intuitive way. We are releasing the equality query type with the 7.0 release and in future releases will add support to the range, prefix, suffix, and substring query types. First announced in preview in MongoDB 6.0 in 2022, Queryable Encryption introduced a fast state-of-the-art encrypted search algorithm using innovative cryptography engineering built and designed by MongoDB’s Cryptography Research Group with decades of experience designing state-of-the-art encrypted search algorithms. Since its initial release last year, MongoDB has worked in partnership with its customers including leading Fortune 500 companies in the healthcare and insurance industries to fine-tune the release for general availability. This client-side encryption approach uses novel encrypted data structures that allow developers to run efficient, expressive queries on encrypted workloads for the first time. Data remains encrypted at all times on the database, including in memory and in the CPU; keys never leave the application and cannot be accessed by the database server. Queryable Encryption: How it works Here is a sample flow of operations where an authorized user wants to query the encrypted data. In this example, let’s assume we are retrieving the records for an SSN number. Authorized users run an equality query to get specific SSN number records Recognizing the query is against an encrypted field, the driver requests the encryption keys from the customer-provisioned key provider, such as AWS Key Management Service (AWS KMS), Google Cloud KMS, Azure Key Vault, or any KMIP-enabled provider, such as HashiCorp Vault. The MongoDB driver gets the encryption keys from the key provider The driver submits the encrypted query along with a cryptographic token to the MongoDB server with the encrypted fields rendered as ciphertext. Queryable Encryption implements a fast encrypted search algorithm that allows the server to process queries on the encrypted data, without knowing the data. The data and the query itself remain encrypted at all times on the server. The MongoDB server returns the encrypted results of the query to the driver. The query results are decrypted with the keys held by the driver and returned to the client and shown as plaintext. Here are some of the key benefits of Queryable Encryption technology: Run equality queries on encrypted data: With Queryable Encryption, customers can run equality queries on encrypted data using a fast state-of-the-art encrypted search algorithm. This algorithm allows the server to process and retrieve matching documents without the server understanding anything about the data or why the document should be returned. Groundbreaking query technology based on standards-based cryptography: Queryable Encryption introduces a fast state-of-the-art encrypted search algorithm that uses NIST standards-based primitives. These are well-tested and established public standards to ensure the confidentiality and integrity of data. Faster application development cycle: Queryable Encryption allows developers to easily encrypt sensitive data without changes to their application code with many language-specific drivers to choose from. There is no crypto experience required and it’s intuitive and easy for developers to set up and use. Developers don't have to figure out how to use the right algorithms, encryption options, etc to implement their right encryption solution. MongoDB has done all that complex work for them. Reduce operational risk as sensitive workloads are protected on the cloud: Eliminate common security concerns when moving database workloads to the cloud. Customers can keep their data on any of the cloud providers and be assured that their data is protected. Since encryption keys are only accessible within the customer environment, the data cannot be decrypted by a 3rd party or the cloud provider. The only place where the data is unencrypted is in the application. Strong technical controls for critical data privacy use cases: Can help customers meet strict data privacy requirements such as HIPAA, GDPR, CCPA, PCI, and more. Queryable Encryption uses strong data protection techniques and end-to-end encryption. Resources For more information on Queryable Encryption, refer to the following resources: Queryable Encryption Documentation Queryable Encryption Quick Start Queryable Encryption FAQ Queryable Encryption Driver Compatibility
4 Big Reasons to Upgrade to MongoDB 7.0
This post is also available in: Deutsch , Français Lately, we've been hitting the road and making news at a series of events in major cities across the globe. One of the big highlights is the release of MongoDB 7.0 , which offers a comprehensive suite of features designed to streamline operations, improve performance, and enhance security. With this release, MongoDB reaffirms itself as a top choice for organizations looking to boost the productivity of their development teams as they build modern, distributed applications. Version 7.0 has all the features released in prior versions with additional features aimed at making it easier for developers to build software. #1 - Enhanced performance MongoDB 7.0 brings significant improvements to working with time series data, especially demanding, high-volume datasets of all shapes. These improvements result in improved storage optimization and compression, as well as improved query performance. Developers will experience even better handling of high cardinality data, improved scalability, and overall performance; enabling you to manage time-series data more efficiently and cost-effectively. Change streams will now support even wider use cases: handling changes in large documents, even with pre-images and post-images, without causing unexpected errors. #2 - Smoother migrations Updates to cluster-to-cluster sync (mongosync) will enable more efficient data migration in a variety of scenarios. Cluster-to-cluster sync now provides greater flexibility in syncing between clusters with unlike topologies, such as from replica sets to sharded clusters. Filtered sync allows for syncing specific data sets instead of the entire cluster. Atlas Live Migrate now supports migrations for clusters running MongoDB 6.0.4+ delivering migrations that are faster and more resilient in cases of interruption during the migration process. #3 - Streamlined developer experience With new enhancements to the aggregation pipeline — including compound wildcard indexes , approximate percentiles, and bitwise operators — developers can enjoy greater flexibility and performance in indexing and querying data. With MongoDB 7.0, developers can also implement user role variables within aggregation pipelines enabling a single view to display different data based on the logged-in users’ permissions. Support for fine-grained updates and deletes in time-series collections and new metrics to help select a shard key to help reduce developer effort and streamline the development process. #4 - Stronger security MongoDB 7.0 strengthens security capabilities with Queryable Encryption to help customers encrypt sensitive data and run equality queries on fully randomized encrypted data. The security enhancements ensure that developers can build and deploy applications with confidence, knowing that their data is protected and compliant with the latest security standards and protocols. Why wait? With a host of new features and enhancements designed to make your team more productive, MongoDB 7.0 is the perfect choice for organizations looking to take their development to the next level. From enhanced performance to stronger security features, MongoDB 7.0 makes it easier to build the next big thing. Register for Atlas now and start building today . If you'd like guidance on upgrading to 7.0, our professional services team offers upgrade support to help ensure a smooth transition. To learn more, see MongoDB Consulting . Check out the full session on What's New in MongoDB 7.0 on our YouTube channel!
Changes to the findOneAnd* APIs in Node.js Driver 6.0.0
Do you use the MongoDB Node.js driver? If so, there’s a good chance you use various find() operations regularly. MongoDB plans to release version 6.0.0 of the Node.js driver in August 2023, and we’ve made some exciting improvements to the findOneAnd* operation. With the new driver release, the modified (or original) document targeted by a findOneAnd* operation will now be returned by default. Current state Up until now, as opposed to returning the requested document, this family of API methods would return a ModifyResult , which would contain the requested document in a value field. This design was due to these APIs leveraging the MongoDB Server’s findOneAndModify command and wrapping the command’s output directly. To demonstrate, let’s adapt the code from the Driver’s documented usage examples to update one document in our movies collection using the findOneAndUpdate API. const database = client.db("sample_mflix"); const movies = database.collection("movies"); // Query for a movie that has the title 'The Room' const query = { title: "The Room" }; const updatedMovie = await movies.findOneAndUpdate(query, { $set: { "imdb.rating": 3.4, "imdb.votes": 25750 } }, { projection: { _id: 0, title: 1, imdb: 1 }, returnDocument: "after" }); console.log(updatedMovie); { lastErrorObject: { n: 1, updatedExisting: true }, value: { title: 'The Room', imdb: { rating: 3.4, votes: 25750, id: 368226 } }, ok: 1, '$clusterTime': { clusterTime: new Timestamp({ t: 1689343889, i: 2 }), signature: { hash: Binary.createFromBase64("3twlRKhDSGIW25WVHZl17EV2ulM=", 0), keyId: new Long("7192273593030410245") } }, operationTime: new Timestamp({ t: 1689343889, i: 2 }) } One of the options we set was a returnDocument of after , which should return the updated document. Though the expectation may be that the function call would return the document directly, as we can see this isn’t the case. While the document you’re looking for can be accessed using updatedMovie.value , that isn’t the most intuitive experience. But changes are on the way! What can we do right now? Starting with the Node.js Driver 5.7.0 release a new FindOneAnd*Options property called includeResultMetadata has been introduced. When this property is set to false (default is true ) the findOneAnd* APIs will return the requested document as expected. const updatedMovie = await movies.findOneAndUpdate(query, { $set: { "imdb.rating": 3.3, "imdb.votes": 25999 } }, { projection: { _id: 0, title: 1, imdb: 1 }, includeResultMetadata: false }); console.dir(updatedMovie); { title: 'The Room', imdb: { rating: 3.3, votes: 25999, id: 368226 } } What about TypeScript? If your application uses TypeScript and the MongoDB Node.js Driver, anywhere a findOneAnd* call is made, if the requested document is required it will be accessed via the value property of the ModifyResult . This occurs when includeResultMetadata is not set or when it is set to true (the current default value). Type hinting will indicate the Schema associated with the collection the operation was executed against. As we would expect, when the includeResultMetadata is changed to false inline validation will indicate there’s an issue as the value property no longer exists on the type associated with the result. Attempting to compile our TypeScript project will also fail. TSError: ⨯ Unable to compile TypeScript: index.ts:31:17 - error TS18047: 'updatedMovie' is possibly 'null'. 31 console.dir(updatedMovie.value); ~~~~~~~~~~~~ index.ts:31:30 - error TS2339: Property 'value' does not exist on type 'WithId<Movie>'. 31 console.dir(updatedMovie.value); Next Steps If you’re using the findOneAnd* family of APIs in your JavaScript or TypeScript project, upgrading the MongoDB Node.js Driver to 5.7.0+ and adding the includeResultMetadata: false option to those API calls will allow you to adapt your application to the new behavior prior to the 6.0.0 release. Once 6.0.0 is released, includeResultMetadata: false will become the default behavior. If your application relies on the previous behavior of these APIs, setting includeResultMetadata: true will allow you to continue to access the ModifyResult directly.