MongoDB Updates
The newest releases and freshest updates
Introducing the Certified MongoDB Atlas Connector for Power BI
This is a collaborative post from MongoDB and Microsoft. We thank Alexi Antonino, Natacha Bagnard, Jad Jarouche from MongoDB, and Bob Zhang, Mahesh Prakriya, and Rajeev Jain from Microsoft for their contributions. Introducing MongoDB Atlas Connector for Power BI, the certified solution that facilitates real-time insights on your Atlas data directly in the Power BI interfaces that analysts know and love! Supporting Microsoft’s Intelligent Data Platform , this integration bridges the gap between Developers and Analytics teams, allowing analysts who rely on Power BI for insights to natively transform, analyze, and share dashboards that incorporate live MongoDB Atlas data. Available in June , the Atlas Power BI Connector empowers companies to harness the full power of their data like never before. Let’s take a deeper look into how the Atlas Power BI Connector can unlock comprehensive, real-time insights on live application data that will help take your business to the next level. Effortlessly model document data with Power Query The Atlas Power BI Connector makes it easy to model document data with native Power BI features and data modeling capabilities. With its SQL-92 compatible dialect, mongosql, you can tailor your data to fit any requirements by transforming heavily nested document data to fit your exact needs, all from your Power Query dashboard. Gain real-time insights on live application data By using the Power BI Connector to connect directly to MongoDB Atlas, you can build up-to-date dashboards in Power BI Desktop and scale insights to your organization through Power BI Service with ease. With no delays caused by data duplication, you can stay ahead of the curve by unlocking real-time insights on Atlas data that are relevant to your business. Empower cross-source data analysis The Power BI Connector's integration with MongoDB Atlas enables you to seamlessly model, analyze, and share insightful dashboards that are built from multiple data sources. By combining Atlas's powerful Data Federation capabilities with Power BI's advanced analytics and visualization tools, you can easily create comprehensive dashboards that offer valuable insights into your data, regardless of where it is stored. See it in action Log in and activate the Atlas SQL Interface to try out the Atlas Power BI Connector ! If you are new to Atlas or Power BI, get started for free today on Azure Marketplace or Power BI Desktop .
The MongoDB for VS Code Extension Is Now Generally Available
Three years ago, we introduced the MongoDB for VS Code Extension to the world in Public Preview. VS Code is the most popular Integrated Development Environment (IDE) for developers, and we were excited to bring the power of MongoDB, one of the world’s most-loved databases, to developers right in their favorite IDE. Since that time, we’ve seen skyrocketing growth in adoption of the extension, which now has over 800k installs and an average rating of 4.5 stars in the VS Code Extension store. The verdict is in: people love not only VS Code and MongoDB, but love a unified experience in the form of the MongoDB for VS Code Extension. Given the popularity of the tool and innovations we’ve continued to make in the experience, we are delighted to announce that the MongoDB for VS Code Extension is now generally available. Why use the extension? This free, downloadable extension makes it easy for developers to build applications and work application data in MongoDB directly from VS Code. Not only do you get the benefit of interacting with MongoDB data in a familiar IDE experience you’ve likely already customized to your preferences—you also can work with your application data and your application code all in one place. And with the extension now generally available (GA), you can have increased confidence in the extension and MongoDB’s long-term commitment to ongoing improvements to the experience. What the extension can do With the MongoDB for VS Code Extension, you get a single unified interface (VS Code) that you already know and love. Within the extension, you can work with your application data from MongoDB side-by-side with your application code for a more streamlined software development experience. Let’s take a look at what you can do with the extension. Connect to MongoDB After you’ve installed the extension , the first thing you’ll want to do is connect to MongoDB using a connection string. If you’re using MongoDB Atlas, you can find your connection string in the Atlas Web UI under the “Database” view by clicking the “Connect” button and then choosing VS Code as your connection option. Data exploration Within the extension, it’s easy to look at your data on MongoDB while working on your code. In the left-hand sidebar, you can easily click through databases, collections, and documents, as well as see relevant schema and indexes. Referencing both schema and indexes here during development can be helpful because: 1. By looking at the schema, you can see what fields you can query on and what their types are, and 2. You can confirm if your query is covered by an index for faster reads against the database. Playgrounds The MongoDB for VS Code Extension gives you a fully-featured JavaScript Playgrounds experience for rapid scripting and prototyping. In Playgrounds you can prototype queries, aggregations, and MongoDB commands with syntax highlighting and intelligent autocomplete. After you write your code, just hit the “play” button or use your favorite keyboard shortcut to instantly see the results of code execution. Within Playgrounds you can: Create new databases and collections Execute Create-Read-Update-Delete (CRUD) operations against your MongoDB database Prototype queries and aggregations using MongoDB’s powerful and expressive Query API Export the syntax for a given query or aggregation to your chosen programming language (including language driver syntax) You can also save Playground files together with your application code and version them in git. This is a great option for documenting all the queries and aggregations your application runs, for scripts that generate or import sample datasets to seed your development clusters, or for scripts that create indexes or define schema migrations. And because Playgrounds use the shell syntax, you can then run them programmatically with the MongoDB Shell. Access the MongoDB Shell Sometimes you just want to run a quick query or command in your terminal rather than using a fully-featured UI. The MongoDB Shell is the perfect tool for these kinds of quick data interactions, and you can access the Shell without ever leaving VS Code. Just right-click on your cluster and select “Launch MongoDB Shell” to get started with the Shell. Terraform If your team uses Terraform, you’ll probably be interested in the MongoDB Atlas Terraform Provider for building with MongoDB. The MongoDB for VS Code Extension gives you access to snippets of code for common tasks you might want to accomplish—including managing your Terraform configuration for Atlas. To use this feature, just open a Terraform file, type atlas , go through the predefined placeholders, and configure your credentials. The MongoDB for VS Code Extension lets you do all of the above - and more. To learn about all the different capabilities of the extension, check out the documentation here . New features Here’s what’s new in the extension now that it’s generally available: Autocomplete support with IntelliSense for using the MongoDB Query API, making it more intuitive to type queries and aggregations for your data on MongoDB Improvements to the Playgrounds experience to make them more reflective of a traditional JavaScript environment, including the ability to integrate them with common tools for the JavaScript ecosystem such as ESLint and Prettier Time series collections can now be created right from Playgrounds You can create column store indexes to support your analytics queries Get started today If you haven’t tried it yet, now is the time to start using the MongoDB for VS Code Extension! To install it, simply search for it in the Extensions list inside VS Code or download it from the VS Code Marketplace . Or if you’re a current user, be sure to check for updates so you get the latest version of the extension and access to the new features that come with it. As you build with the MongoDB for VS Code Extension, feel free to give us feedback on your product experience in the MongoDB Feedback Engine , so we can continue to take the pulse of the community and further optimize the extension for users.
MongoDB Atlas Expands Globally with AWS
We’re proud to announce our global expansion of MongoDB Atlas on AWS (Amazon Web Services) in the Middle East, Europe, and APAC. The launch of regions in the United Arab Emirates (UAE), Zurich, Spain, Hyderabad, and Melbourne expands availability of MongoDB Atlas to 27+ AWS regions around the world. The UAE region is an AWS Recommended Region , meaning it has three Availability Zones (AZ), bringing significant infrastructure to the Middle East. When you deploy a cluster in the UAE, Atlas automatically distributes replicas to the different AZs for higher availability. If there’s an outage in one zone, the Atlas cluster will automatically fail over to keep running in the other two. And you can also deploy multi-region clusters with the same automatic failover built-in. We’re delighted that — as with customers in Bahrain, Cape Town, and more — United Arab Emirates organizations will now be able to keep data in their own country, delivering low-latency performance and ensuring confidence in data locality. UAE customers in government, financial services, and utilities in particular will benefit from this expansion. In addition to the launch in the UAE region, MongoDB Atlas is now available in Zurich and Spain, expanding to our already strong presence in the EMEA and giving our customers the ability to build and run applications with data sovereignty requirements for the region. MongoDB was awarded AWS Marketplace Partner of the Year - EMEA for 2022, and we are committed to continuing to make Atlas easily accessible across the region. Our expansion in APAC is also particularly exciting given the recent momentum of MongoDB Atlas on AWS in the region. Increased availability in India and Australia will help to secure the opportunity for APAC developers to have wider access to build with high performance. Companies like Open Government Products, Bendigo&Adelaide, Cathay Pacific, Dongwha, and Kasikorn will benefit from closer availability zones. We’re confident our developers around the world will appreciate this capability as they build tools to improve citizens’ lives and better serve their local users. Get started with MongoDB Atlas for free today on AWS Marketplace Learn more about MongoDB Atlas on AWS
What's New in Atlas Charts: Suggested Charts, Auto Activation, and Contextual Help
Atlas Charts is the native data visualization tool for quickly and easily analyzing your data in MongoDB Atlas. Today, we’re announcing a collection of updates that further streamline the Charts experience: Suggested charts: a quicker way to build visualizations More contextual help in the chart builder Automatic Charts activation for all project members Suggested charts Charts has always offered a simple UI with an easy to use, drag and drop interface that lets you quickly build charts and visualize your application data. However, we still found that some users could benefit from extra help when building out new charts. Rather than starting from an empty screen where you need to drag appropriate fields into the chart type selected, what if you could simply select an automatically suggested chart, and start applying customization from there? That’s exactly what suggested charts offer. We experimented with this feature late last year and now we have turned it on for everyone! Simply add a chart into one of your dashboards to try it out today. Figure 1: Using the new auto suggested charts in the chart builder. Help button in the chart builder As you might expect, the chart builder is where you do your chart creation. Similar to suggested, last year we experimented with ways to provide more contextual help for users when building new charts. Now, we are surfacing helpful docs articles to educate users on key chart building topics like: filtering, adding fields, selecting the right chart type, and more. Sometimes it can be intimidating to know exactly what chart to use and how to achieve the style and customization you want – the help button in chart builder will make this much easier. Building a chart and have a question? Just click into the Get help button and check out one of our highlighted topics, or choose View all topics to read the main Charts documentation. Streamlining Charts activation We’re constantly looking for ways to help Atlas users with data visualization quicker. So starting with this latest release, when you click into the Charts tab from the Atlas UI, you will automatically be set up to start visualizing your data – no activation required. Additionally, Atlas users browsing collections within a specific cluster, can now more quickly navigate directly into Charts for quick visualization. When viewing a collection, the Visualize your data button, seamlessly opens in the chart builder with the current collection selected as the chart’s data source. Paired with the new suggested chart, users see a list of chart suggestions to quickly and easily build a relevant chart based on their collection data. Note: you may see a “Charts” tab in the collection view instead of the Visualize your data button, as shown below. This is due to an experiment we are currently running. FIgure 3: Seamlessly navigate from an Atlas collection into the chart builder in Atlas Charts. This is a continuation of our effort to optimize the overall Charts experience. Last year we made strides in this area by introducing features like streamlined data sources and org-wide sharing . Keep on the lookout for more Charts features that further simplify your experience visualizing Atlas data across your team. New to Atlas Charts? Get started today by logging into or signing up for MongoDB Atlas , deploying or selecting a cluster, and activating Charts for free.
What's New in Atlas Charts: Schedule Dashboard Reports to Share Data with Your Team
Today, we’re introducing an exciting feature addition for teams using Atlas Charts . Charts project owners can now schedule dashboard reports to be sent via email to keep team members informed about key data. This feature has been heavily requested by some of our largest users as there are many use cases where dashboards may be valuable to your team, but you don’t necessarily want to require anyone to do extra work to access and view data. Enter scheduled dashboard reports in Atlas Charts! In any dashboard that your team relies on for regular data review, simply schedule a dashboard report. The new Schedule button can be found at the top right of the dashboard screen: Once you’ve chosen a dashboard from which to create a report, you will see a variety of options letting you customize the content and frequency of your report before you schedule. A report requires basic fields like a name or subject line, recipient list, and optionally, a message for the body of the email. In addition to a link to the dashboard in Charts, you can choose whether to attach an image or PDF for quick reference in the message itself. Finally, you can set a schedule of daily, weekly, monthly, or quarterly delivery. You can also simply send a single email if you have a one-time need to share a report. And once you’ve set everything up, your email will be sent on your defined schedule. As you use scheduled dashboard reports more and more, we created a Reports page where you can manage all reports in your project. Note that if you’re on an free tier, you can try one scheduled report. If you’re on an M2 cluster or higher, you can create up to 100 reports per project. To learn more, please check out our documentation . We’re always listening to feature requests that will enhance using Charts across teams, so if you have any requests or feedback, please share them with us here . Log in to Atlas Charts today to schedule your first report! If you’re new to Atlas Charts, get started today by logging into or signing up for MongoDB Atlas.
MongoDB Releases “Focus Mode” in Compass GUI
We’re excited to announce an improvement to the aggregation-building experience in MongoDB Compass. Compass already makes it easy to view and manage your MongoDB databases, and with the addition of Focus Mode you now have the option to dial in on specific stages within your aggregation pipeline. Overview MongoDB's Query API and Aggregation Pipelines enable easy retrieval and processing of data from collections. They also facilitate complex operations such as filtering, grouping, and transforming, making computation and analysis effortless. MongoDB Compass' intuitive interface simplifies the process of building aggregations by enabling developers to easily create, test, and refine aggregation pipelines, and the introduction of Focus Mode takes this a step further. When constructing pipelines, having to simultaneously view and consider multiple stages can make it challenging to analyze the impact of a specific stage, leading to increased cognitive load. Now, developers can toggle Focus Mode on stages, opening a view that focuses exclusively on the contents of the specific stage they are working on. This view can also be used to view sample input (before the aggregation stage is applied) and output (after the stage is applied) documents, aiding in the understanding, troubleshooting, and optimizing of the data pipeline. Developers can also switch between different stages by accessing a drop-down menu at the top of their screen. This makes identifying inefficiencies and optimizing performance easier, as well as providing deeper insights from the output documents for data-driven decision making. Focus Mode offers a streamlined and distraction-free environment for working with stages, improving the efficiency and precision of testing, debugging, and analyzing the impact of each stage on the data, ultimately simplifying the creation and management of pipelines. Conclusion The addition of Focus Mode is part of our continued refresh of the query and aggregation experience in Compass. These improvements are made possible thanks to the feedback of our developer community, so we encourage you to try out this new feature and let us know what you think! To learn more about Aggregation Pipeline Builder in Compass, visit our documentation .
Visualizing Your MongoDB Atlas Data with Atlas Charts
MongoDB Atlas is the leading multi-cloud developer data platform. We see some of the world’s largest companies in manufacturing , healthcare , telecommunications , and financial services all build their businesses with Atlas at their foundation. Every company comes to MongoDB with a need to safely store operational data. But all companies also have a need to analyze data to gain insights into their business and data visualization is core to establishing that real-time business visibility. Data visualization enables the insights required to take action, whether that’s on key sales data, production and operations data, or product usage to improve your applications. The best way to do this as an Atlas user is by using Atlas Charts – MongoDB’s first-class data visualization tool, built natively into MongoDB Atlas. Why choose Charts First, Charts is natively built for the document model. If you’re familiar with MongoDB, you should be familiar with documents. The document model is a data model made for the way developers think. And with Charts, you can take your data from documents and collections in Atlas, and visualize them with no ETL, data movement or duplication. This speeds up your ability to discover insights. Second, Charts supports all cluster configurations you can create in Atlas, including dedicated clusters, serverless instances, data stored in Online Archive, as well as federated data in Atlas Data Federation. Typically when you learn about a company’s integrated products and services, you find some “gotchas” or limitations that make any benefits come at a significant cost. In the case of a MongoDB Atlas customer, that could come in the form of someone finding out that a cluster configuration option isn’t supported by Charts. But that will never be the case. If you create and manage your application data in Atlas, you can visualize it in Charts. That’s it. Third, Charts is a robust data visualization tool with a variety of chart types, extensive customization options, and interactivity. Compared to other options in the business intelligence market, you get the same key benefits, without all the complexity. You can learn how to use Charts in a few hours and you can easily teach your team. It’s the simplest data visualization solution for most teams. Fourth, the value of Charts can extend beyond individual use cases, with sharing and embedding . This lets you both flexibly share charts and dashboards with your team, as well as embed them into contexts that matter most to your data consumers, such as in a blog post or inside your company’s wiki. Finally, Charts is free for Atlas users up to 1GB per project per month, which covers moderate usage for most teams. There are no seat-based licensing fees associated with Charts, so no matter how many team members you have, Charts will remain a low-cost, if not zero cost solution for your data visualization needs. Beyond the included free usage, it’s just $1/GB transferred per month. You can check out more pricing details here . How to use Charts The best way to learn how to use Charts is to simply give it a try. It’s free to use and we have a variety of sample dashboards you can use to get started. But let’s walk through some basics to help illustrate the kinds of visualizations that Charts can enable. Charts makes visualizing your data easy by automatically making your Atlas deployments (any cluster configuration) available for visualization. If you’re a project owner, you can manage permissions to data sources in Charts. We could write an entire blog post on data sources, but if you’re just getting started, just know that your data is made easily available in Charts unless your project owner intentionally hides it. Create a dashboard Everything in Charts starts with a dashboard and creating a dashboard is easy. Simply select the Add Dashboard button at the top right of the Charts page in Atlas . From there, you’ll fill in some basic information like a title and optional description, and you’re on your way. Here’s what one of our new sample dashboards looks like. They are a great place to start: Build a chart Once you have a dashboard created, you can add your first chart. The chart builder gives you a simple and powerful drag and drop interface to help you quickly construct charts. The first step is selecting your data source: Once you have a data source selected, simply add desired fields into your chart and start customizing. The example below uses our IoT sample dashboard dataset to create a bar chart displaying the total distance traveled by different users. From there you can add filters and further customize your chart by adding custom colors, data labels, and more. The chart builder even allows you to write, save, and share queries and aggregation pipelines as shown below. You can learn more in our documentation. Play around with the chart builder to get familiar with all of its functionality. Share and embed A chart can be useful in itself to individual users, but we see users get the most benefit out of Charts when sharing visualizations with others. Once you have created a dashboard with one or more charts, we offer a variety of options letting you share your dashboards with your team, your organization, or via a public link if your data is not sensitive. If you would rather embed a chart or dashboard where your team is already consuming information, check out Charts embedding functionality. Charts lets you embed a chart or dashboard via iframe or SDK, depending on your use case. Check out our embedding documentation to learn more. That was just a brief overview of how to build your first charts and dashboards in Atlas Charts, but there’s a lot more functionality to explore. For a full walkthrough, watch our product demo here: Atlas Charts is the only native data visualization tool built for the document model and it’s the quickest and easiest way to get started visualizing data from Atlas. We hope this introduction helps you get started using Charts to gain greater visibility into your application data, helping you to make better decisions on your data. Get started with Atlas Charts today by logging into or signing up for MongoDB Atlas , deploying or selecting a cluster, and navigating to the Charts tab to activate for free.
MongoDB Atlas Integrations for CDKTF are now Generally Available
Infrastructure as Code (IaC) tools allows developers to manage and provision infrastructure resources through code, rather than through manual configuration. IaC have empowered developers to apply similar best practices from software development to application instructure deployments. This includes: Automation - helping to ensure repeatable, consistent, and reliable infrastructure deployments Version Control - check in IaC code into GitHub, BitBucket, or GitLab for improved team collaboration and higher code quality Security - create clear audit trails of each infrastructure modification Disaster Recovery - IaC scripts can be used to quickly recreate infrastructure in the event of availability zone or region outages Cost Savings - prevent overprovisioning and waste of cloud resources Improved Compliance - easier to enforce organizational policies and standards Today we are doubling down on this commitment and announcing MongoDB Atlas integrations with CDKTF (Cloud Development Kit for Terraform). These new integrations are built on top of the Atlas Admin API and allow users to automate infrastructure deployments by making it easy to provision, manage, and control Atlas infrastructure as code in the cloud without first having to create in HCL or YAML configuration scripts. CDKTF abstracts away the low-level details of cloud infrastructure, making it easier for developers to define and manage their infrastructure natively in their programming language of choice. Under the hood, CDKTF is converted into Terraform config files on your behalf. This helps to simplify the deployment process and eliminates context switching. MongoDB Atlas & HashiCorp Terraform: MongoDB began this journey with our partners at HashiCorp when we launched the HashiCorp Terraform MongoDB Atlas Provider in 2019. We then have since grown to 10M+ downloads over all time and our provider is the number one provider in the database category. Today we are delighted to support all CDKTF supported languages including JavaScript, TypeScript, Python, Java , Go, and .NET. In addition, with CDKTF users are free to deploy their MongoDB Atlas resources to AWS, Azure and Google Cloud enabling true multi-cloud deployments. Learn how to get started via this quick demo . Start building today! MongDB Atlas CDKTF integrations are free and open source licensed under Mozilla Public License 2.0 . Users only pay for underlying Atlas resources created and can get started with Atlas always free tier ( M0 clusters ). Getting started today is faster than ever with MongoDB Atlas and CDK for HashiCorp Terraform . We can’t wait to see what you will build next with this powerful combination! Learn more about MongoDB Atlas and CDK for Hashicorp Terraform
MongoDB Atlas Integrations for AWS CloudFormation and CDK are now Generally Available
Infrastructure as Code (IaC) tools allows developers to manage and provision infrastructure resources through code, rather than through manual configuration. IaC have empowered developers to apply similar best practices from software development to application instructure deployments. This includes: Automation - helping to ensure repeatable, consistent, and reliable infrastructure deployments Version Control - check in IaC code into GitHub, BitBucket, AWS CodeCommit, or GitLab for improved team collaboration and higher code quality Security - create clear audit trails of each infrastructure modification Disaster Recovery - IaC scripts can be used to quickly recreate infrastructure in the event of availability zone or region outages Cost Savings - prevent overprovisioning and waste of cloud resources Improved Compliance - easier to enforce organizational policies and standards Today we are doubling down on this commitment and announcing MongoDB Atlas integrations with AWS CloudFormation and Cloud Development Kit (CDK). AWS CloudFormation allows customers to define and provision infrastructure resources using JSON or YAML templates. CloudFormation provides a simple way to manage infrastructure as code and automate the deployment of resources. AWS Cloud Development Kit (CDK) is an open-source software development framework that allows customers to define cloud infrastructure in code and provision it through AWS CloudFormation. It supports multiple programming languages and allows customers to use high-level abstractions to define infrastructure resources. These new integrations are built on top of the Atlas Admin API and allow users to automate infrastructure deployments by making it easy to provision, manage, and control Atlas Infrastructure as Code in the cloud. MongoDB Atlas & AWS CloudFormation: To meet developers where they are, we now have multiple ways to get started with MongoDB Atlas using AWS Infrastructure as Code. Each of these allow users to provision, manage, and control Atlas infrastructure as code on AWS: Option 1: AWS CloudFormation Customers can begin their journey using Atlas resources directly from the AWS CloudFormation Public Registry . We currently have 33 Atlas Resources and will continue adding more. Examples of available Atlas resources today include: Dedicated Clusters, Serverless Instances, AWS PrivateLink , Cloud Backups, and Encryption at Rest using Customer Key Management. In addition, we have published these resources to 22 (and counting) AWS Regions where MongoDB Atlas is supported today. Learn how to get started via this quick demo . Option 2: AWS CDK After its launch in 2019 as an open source project, AWS CDK has gained immense popularity among the developer community with over a thousand external contributors and more than 1.3 million weekly downloads. AWS CDK abstracts away the low-level details of cloud infrastructure, making it easier for developers to define and manage their infrastructure natively in their programming language of choice. This helps to simplify the deployment process and eliminates context switching. Under the hood, AWS CDK synthesizes CloudFormation templates on your behalf which is then deployed to AWS accounts. In AWS CDK, L1 (Level 1) and L2 (Level 2) constructs refer to two different levels of abstraction for defining infrastructure resources: L1 constructs are lower-level abstractions that provide a one-to-one mapping to AWS CloudFormation resources. They are essentially AWS CloudFormation resources wrapped in code, making them easier to use in a programming context. L2 constructs are higher-level abstractions that provide a more user-friendly and intuitive way to define AWS infrastructure. They are built on top of L1 constructs and provide a simpler and more declarative API for defining resources. Today we announce MongoDB Atlas availability for AWS CDK in JavaScript and TypeScript, with plans for Python, Java, Go, and .NET support coming later in 2023. Now customers can easily deploy and manage all available Atlas resources by vending AWS CDK applications with prebuilt L1 Constructs. We also have a growing number of L2 and L3 CDK Constructs available. These include Constructs to help users to quickly deploy the core resources they need to get started with MongoDB Atlas on AWS in just a few lines JavaScript or TypeScript (see awscdk-resources-mongodbatlas to learn more). Users can also optionally select to add more advanced networking configurations such as VPC peering and AWS PrivateLink. Option 3: AWS Partner Solutions (previously AWS Quick Starts) Instead of manually pulling together multiple Atlas CloudFormation resources, AWS Partner Solutions gives customers access to pre-built CloudFormation templates for both general and specific use cases with MongoDB Atlas. By using AWS Partner Solution templates, customers can save time and effort compared to architecting their deployments from scratch. These were jointly created and incorporate best practices from MongoDB Atlas and AWS. Go to the AWS Partner Solutions Portal to get started. Start building today! These MongDB Atlas integrations with AWS CloudFormation are free and open source licensed under Apache License 2.0 . Users only pay for underlying Atlas resources created and can get started with Atlas always free tier ( M0 clusters ). Getting started today is faster than ever with MongoDB Atlas and AWS CloudFormation. We can’t wait to see what you will build next with this powerful combination! Learn more about MongoDB Atlas integrations with AWS CloudFormation
What’s New in Atlas Charts: New Sample Dashboards and Embedding SDK Improvements
Atlas Charts is the best data visualization tool for quickly and easily analyzing all your Atlas data. Today, we’re announcing two big updates: New sample dashboards to give you even more inspiration when getting started Charts Embedding SDK enhancements: dashboard filters and save as PNG Let's start with sample dashboards We took a look at the current experience for new Charts users building their first dashboards. To help reduce the time it takes to experience the power of dashboards, we have always offered a sample dashboard that used a movie database to help explore Charts basics like chart types and understanding how a data source and fields work. While any dataset can be interesting and most of us enjoy watching movies, we knew that most development teams could benefit from some tangible examples of how they might visualize their application data in Charts. So we thought about a few key use cases that nearly every team must consider and we put together four new sample dashboards analyzing: product usage, sales or general business metrics, IoT sensor data, and finally, observability or log data commonly used to understand a platform’s reliability. All of these new dashboards are now available to every Charts user, and each provides a great starting point for exploring your data in Charts. Choosing the new sample IoT dashboard in Atlas Charts. With these new sample dashboards as inspiration, you will be able to quickly see the potential of complex dashboards and customize them to fit your own data and gain valuable business insights. Embedding SDK enhancements: Dashboard filters and save as PNG Next, we are continually listening to customers and trying to make our Embedding SDK more useful for diverse use cases. If you’re unfamiliar, the Embedding SDK lets you take charts and dashboards from Charts, and embed them into the contexts that matter most to your users, with rich customization. Up to this point, dashboard filters have only functioned in dashboards built within the Charts UI in MongoDB Atlas. Dashboard filters can be used to dig a level deeper into your datasets, and with interactive filtering , they are the foundation from which you can add interactivity into embedded dashboards. With this update, users can now seamlessly filter data directly within the embedded dashboard , providing a more interactive experience and enhancing data exploration. Setting up dashboard filters in the UI is simple, and once they’re enabled, you can either selectively choose the fields allowed for filtering or allow all fields present in the data sources used in the dashboard. You can also allow all fields present in a data source if you use chart embedding. Configuring an embedded dashboard with dashboard filters in the IoT sample dashboard. Taking the field ‘calories’ as an example used in the video above, in the SDK, you can easily set a dashboard filter to plot a chart with users who have spent more than 1000 calories: dashboard.setFilter({ calories: { $gt: 1000 } }); You can also use the getFilter method to see what filters have been applied to the embedded dashboard: dashboard.getFilter(); Additionally, you can now programmatically save an image of any chart as a PNG, in either base64 or binary formats using the getImage method in the SDK. We are excited to see how you use the sample dashboards and dashboard filtering in embedded dashboards to achieve your visualization goals using Atlas Charts. New to Atlas Charts? Get started today by logging into or signing up for MongoDB Atlas , deploying or selecting a cluster, and activating Charts for free.
New Aggregation Pipeline Text Editor Debuts in MongoDB Compass
There’s a reason why Compass is one of MongoDB’s most-loved developer tools: because it provides an approachable and powerful visual user interface for interacting with data on MongoDB. As part of this, Compass’s Aggregation Pipeline Builder abstracts away the finer points of MongoDB’s Query API syntax and provides a guided experience for developing complex queries. But what about when you want less rather than more abstraction? That’s where our new Aggregation Pipeline Text Editor comes in. Recently released on Compass, the Aggregation Pipeline Text Editor allows users to write free-form aggregations. While users could previously write and edit pipelines through a guided and structured builder organized by aggregation stage, a text-based builder can be preferable for some users. This new pipeline editor makes it easy for users to: See the entire pipeline without having to excessively scroll through the UI Stay “in the flow” when writing aggregations if they are already familiar with MongoDB’s Query API syntax Copy and paste aggregations built elsewhere (like in MongoDB’s VS Code Extension ) into Compass Use built-in syntax formatting to make pipeline text “pretty” before copying it over from Compass to other tools The Aggregation Pipeline Text Editor in Compass. Notice how toward the top right you can click on “stages” to move back to the traditional stage-based Aggregation Pipeline Builder. Ultimately, the addition of the Aggregation Pipeline Text Editor to Compass gives users more flexibility depending on how they want to build aggregations. For a more guided experience and to get result previews when adding each new stage, the existing Aggregation Pipeline Builder will work best for most users. But when writing free-form aggregations or copying and pasting aggregation text from other tools, the Aggregation Pipeline Text Editor may be preferable. It also previews the final pipeline output, rather than the stage-by-stage preview that exists today. Users will be able to access either both the traditional Aggregation Pipeline Builder and the new Pipeline Text Editor from directly within the Aggregations tab in Compass and can switch between the two views without losing their work. To get access to the new Aggregation Pipeline Text Editor, make sure to download the latest version of Compass here . And as always, we welcome your continued feedback on how to improve Compass. If you have ideas for how to improve your experience with Compass you can submit them on our UserVoice platform here . We’ll have even more great features coming in Compass soon. Keep checking back on our blog for the latest news!
Introducing MongoDB Connector for Apache Kafka version 1.9
Today, MongoDB released version 1.9 of the MongoDB Connector for Apache Kafka! This article highlights the key features of this new release! Pre/Post document states In MongoDB 6.0, Change Streams added the ability to retrieve the before and after state of an entire document . To enable this functionality on the collection you can set it as a parameter in the createCollection command such as: db.createCollection( "temperatureSensor", { changeStreamPreAndPostImages: { enabled: true } } ) Alternatively, for existing collections, use colMod as shown below: db.runCommand( { collMod: <collection>, changeStreamPreAndPostImages: { enabled: <boolean> } } ) Once the collection is configured for pre and post images, you can set the change.stream.full.document.before.change source connector parameter to include this extra information in the change event. For example, consider this source definition: { "name": "mongo-simple-source", "config": { "connector.class": "com.mongodb.kafka.connect.MongoSourceConnector", "connection.uri": "<< MONGODB CONNECTION STRING >>", "database": "test", "collection": "temperatureSensor", "change.stream.full.document.before.change":"whenavailable" } } When the following document is inserted: db.temperatureSensor.insertOne({'sensor_id':1,'value':100}) Then an update is applied: db.temperatureSensor.updateOne({'sensor_id':1},{ $set: { 'value':105}}) You can see the change stream event written to Kafka topic is as follows: { "_id": { "_data": "82636D39C8000000012B022C0100296E5A100444B0F5E386F04767814F28CB4AAE7FEE46645F69640064636D399B732DBB998FA8D67E0004" }, "operationType": "update", "clusterTime": { "$timestamp": { "t": 1668102600, "i": 1 } }, "wallTime": { "$date": 1668102600716 }, "ns": { "db": "test", "coll": "temperatureSensor" }, "documentKey": { "_id": { "$oid": "636d399b732dbb998fa8d67e" } }, "updateDescription": { "updatedFields": { "value": 105 }, "removedFields": [], "truncatedArrays": [] }, "fullDocumentBeforeChange": { "_id": { "$oid": "636d399b732dbb998fa8d67e" }, "sensor_id": 1, "value": 100 } } Note the fullDocumentBeforeChange key includes the original document before the update occurred. Starting the connector at a specific time Prior to version 1.9, when the connector starts as a source, it will open a MongoDB change stream and any new data will get processed by the source connector. To copy all the existing data in the collection first before you begin processing the new data, you specify the “ copy.existing ” property. One frequent user request is to start the connector based upon a specific timestamp versus when the connector starts. In 1.9 a new parameter called startup.mode was added to specify when to start writing data. startup.mode=latest (default) “Latest” is the default behavior and starts processing the data when the connector starts. It ignores any existing data when the connector starts. startup.mode=timestamp “timestamp” allows you to start processing at a specific point in time as defined by additional startup.mode.timestamp.* properties. For example, to start the connector from 7AM on November 21, 2022, you set the value as follows: startup.mode.timestamp.start.at.operation.time=’2022-11-21T07:00:00Z’ Supported values are an ISO-8601 format string date as shown above or as a BSON extended string format. startup.mode=copy.existing Same behavior as the existing as the configuration option, “copy.existing=true”. Note that “copy.existing” as a separate parameter is now deprecated. If you defined any granular copy.existing parameters such as copy.existing.pipeline, just prepend them with “startup.mode.copy.existing.” property name. Reporting MongoDB errors to the DLQ Kafka supports writing errors to a dead letter queue . In version 1.5 of the connector, you could write all exceptions to the DLQ through the mongo.error.tolerance=’all’ . One thing to note was that these errors were Kafka generated errors versus errors that occurred within MongoDB. Thus, if the sink connector failed to write to MongoDB due to a duplicate _id error, for example, this error wouldn’t be written to the DLQ. In 1.9, errors generated within MongoDB will be reported to the DLQ. Behavior change on inferring schema Prior to version 1.9 of the connector, if you are inferring schema and insert a MongoDB document that contains arrays with different value data types, the connector is naive and would simply set the type for the whole array to be a string. For example, consider a document that resembles: { "myfoo": [ { "key1": 1 }, { "key1": 1, "key2": "dogs" } ] } If we set output.schema.infer.value . to true on a source connector, the message in the Kafka Topic will resemble the following: … "fullDocument": { … "myfoo": [ "{\"key1\": 1}", "{\"key1\": 1, \"key2\": \"dogs\"}" ] }, … Notice the array items contain different values. In this example, key1 is a subdocument with a single value the number 1, the next item in the “myfoo” array is a subdocument with the same “key1” field and value of an integer, 1, and another field, “key 2” that has a string as a value. When this scenario occurs the connector will wrap the entire array as a string. This behavior can also apply when using different keys that contain different data type values. In version 1.9, the connector when presented with this configuration will not wrap the arrays, rather it will create the appropriate schemas for the variable arrays with different data type values. The same document when run in 1.9 will resemble: "fullDocument": { … "myfoo": [ { "key1": 1, }, { "key1": 1, "key2": "DOGS" } ] }, Note that this behavior is a breaking change and that inferring schemas when using arrays can cause performance degradation for very large arrays using different data type values. Download the latest version of the MongoDB Connector for Apache Kafka from Confluent Hub! To learn more about the connector read the MongoDB Online Documentation . Questions? Ask on the MongoDB Developer Community Connectors and Integrations forum!