MongoDB Updates
The newest releases and freshest updates
MongoDB Atlas for Government Achieves "FedRAMP In-process"
We are pleased to announce that MongoDB Atlas for Government has achieved the FedRAMP designation of “ In-process ”. This status reflects MongoDB’s continued progress toward a FedRAMP Authorized modern data platform for the US Government. Earlier this year, MongoDB Atlas for Government achieved the designation of FedRAMP Ready . MongoDB is widely used across the Federal Government, including the Department of Veterans Affairs, the Department of Health & Human Services (HHS), the General Services Administration, and others. HHS is also sponsoring the FedRAMP authorization process for MongoDB. What is MongoDB Atlas for Government? MongoDB Atlas for Government is an independent environment of our flagship cloud product MongoDB Atlas. Atlas for Government has been built for US government needs. It allows federal, state, and local governments as well as educational institutions to build and iterate faster using a modern database-as-a-service platform. The service is available in AWS GovCloud (US) and AWS US East/West regions. MongoDB Atlas for Government Highlights: Atlas for Government clusters can be created in AWS GovCloud East/West or AWS East/West regions. Atlas for Government clusters can span regions within AWS GovCloud or within AWS. Atlas core features such as automated backups, AWS PrivateLink, AWS KMS, federated authentication, Atlas Search, and more are fully supported Applications can use client-side field level encryption with AWS KMS in GovCloud or AWS East/West. Getting started and pricing MongoDB Atlas for Government is available to Government customers or companies that sell to the US Government. You can buy Atlas for Government through AWS GovCloud or the AWS marketplace . Please fill out this form and a representative will get in touch with you. To learn more about Atlas for Government, visit the product page , check out the documentation , or read the FedRAMP FAQ .
Serverless Instances Now Offer Extended Regional and Cloud Provider Support
Today’s applications are expected to just work, regardless of time of day, user traffic, or where in the world they are being accessed from. But in order to achieve this level of performance and scale, developers have to meticulously plan for infrastructure needs, sometimes before they even know what the success of their application may be. In many cases, this is not feasible and can lead to over provisioning and over paying. But what if you could forgo all of this planning and the database would seamlessly scale for you? Well, now you can - with serverless instances on MongoDB Atlas. Since we announced serverless instances in preview at MongoDB.live we have been actively working toward implementing new functionality to make them more robust and widely available. With our most recent release, serverless instances now offer expanded cloud providers and regions, and support MongoDB tools. Deploy a serverless instance on the cloud provider of your choice With our dedicated clusters on MongoDB Atlas, you have the flexibility to run anywhere with global reach on the cloud provider of your choice, so you can deliver responsive and reliable applications wherever your users are located. Our goal is to provide this same flexibility for serverless instances. We’re happy to announce that you can now deploy a serverless instance in ten regions on AWS, Google Cloud, and Azure. You’ll see when deploying a serverless instance there are now more regions supported on AWS, as well as two available regions on both Google Cloud and Azure - so you can get started with the cloud provider that best suits your needs or the region that’s closest to you. We will be continuing to add new regions over time to ensure coverage where you need it most. Easily import your data with MongoDB tools With this release, we have also made it easier to work with your data. You can now easily import data from an existing MongoDB deployment using the MongoDB Tools including mongodump, mongorestore, mongoexport , and mongoimport . In order to use MongoDB tools with serverless instances, you will need to be using the latest version . If you have additional feature requests that would make your developer experience better, share them with us in our feedback forums . Database deployment made simple With serverless instances, you can get started with almost no configuration needed - MongoDB Atlas will automatically scale to meet your workload needs, whether you have variable traffic patterns or you’re looking for a sandbox database for your weekend hobby project. If you haven’t yet given serverless instances a try, now is a great time to see what they can offer. If you have feedback or questions, we’d love to hear them! Join our community forums to meet other MongoDB developers and see what they’re building with serverless instances. Create your own serverless instance on MongoDB Atlas. Try the Preview .
Highlight What Matters with the MongoDB Charts SDK
We're proud to announce that with the latest release of the MongoDB Charts SDK you can now apply highlights to your charts. These allow you to emphasize and deemphasize your charts with our MongoDB query operators . Build a richer interactive experience for your customers by highlighting with the MongoDB Charts embedding SDK . By default, MongoDB Charts allows for emphasizing parts of your charts by series when you click within a legend. With the new highlight capability in the Charts Embedding SDK, we put you in control of when this highlighting should occur, and what it applies to. Why would you want to apply highlights? Highlighting opens up the opportunity for new experiences for your users. The two main reasons why you may want to highlight are: To show user interactions: We use this in the click handler sandbox to make it obvious what the user has clicked on. You could also use this to show documents affected by a query for a control panel. Attract the user’s attention: If there's a part of the chart you want your users to focus on, such as the profit for the current quarter or the table rows of unfilled orders. Getting started With the release of the Embedding SDK , we've added the setHighlight method to the chart object, which uses MQL queries to decide what gets highlighted. This lets you attract attention to marks in a bar chart, lines in a line chart, or rows in a table. Most of our chart types are already supported, and more will be supported as time goes on. If you want to dive into the deep end, we've added a new highlighting example and updated the click event examples to use the new highlighting API: Highlighting sandbox Click events sandbox Click events with filtering sandbox The anatomy of a click In MongoDB Charts, each click produces a wealth of information that you can then use in your applications , as seen below: In particular, we generate an MQL expression that you can use called selectionFilter , which represents the mark selected. Note that this filter uses the field names in your documents, not the channel names. Before, you could use this to filter your charts with setFilter , but now you can use the same filter to apply emphasis to your charts. All this requires is calling setHighlight on your chart with the selectionFilter query that you get from the click event, as seen in this sandbox . Applying more complex highlights Since we accept a subset of the MQL language for highlighting, it's possible to specify highlights which target multiple marks, as well as multiple conditions. We can use expressions like $lt and $gte to define ranges which we want to highlight. And since we support the logical operators as well, you can even use $and / $or . All the Comparison , Logical and Element query operators are supported, so give it a spin! Conclusion This ability to highlight data will make your charts more interactive and help you better emphasize the most important information in your charts. Check out the embedding SDK to start highlighting today! New to Charts? You can start now for free by signing up for MongoDB Atlas , deploying a free tier cluster and activating Charts. Have an idea on how we can make MongoDB Charts better? Feel free to leave an idea at the MongoDB Feedback Engine .
Fine-Tune Relevance in MongoDB Atlas Search with Function Scoring and Synonyms
MongoDB Atlas Search is an embedded full-text search solution in MongoDB Atlas that gives developers a seamless and scalable experience for building fast, relevance-based application features. We announced its general availability last year at MongoDB.live 2020 and over the past year we’ve introduced many new features, including a visual index builder, search query tester, custom analyzers , and wildcard path queries . This year at MongoDB.live 2021 , we’re excited to highlight two new capabilities that help developers tune the relevance of search results. See how easy it is to get started with MongoDB Atlas Search in this demo video by Marcus Eagan, Senior Product Manager for Atlas Search. Building relevance into search results Understanding the behavior of your users is essential when thinking about search result relevance. People don’t always tell you what they want, and they sometimes use words or phrases that don’t match your content exactly. To cover these scenarios, you can use full-text search features like function scoring and synonyms. Influence search rankings with function scoring There are often multiple factors that influence how search results should be ranked. For example, let’s say you have a restaurant finder application. The explicit inputs are things like the user’s location and what they’re searching for, but what’s implied is that they likely want to see highly rated restaurants or ones with more reviews. What’s Cooking: a sample restaurant finder application using MongoDB Atlas Search Function scoring allows you to influence the order of results returned by manipulating the score of each result. In Atlas Search, that means you can use a numeric field in a document and apply a mathematical expression to it. For example, you might want to increase the score of restaurants that are sponsored or have higher star ratings. This can easily be accomplished within the same search query by simply adding the function option to the score parameter of your query. Learn more about how to use function scores in our developer tutorial . Show results for more search queries with synonyms Synonyms are often used to define terms that are semantically similar to each other to improve search results. For example, someone searching for “noodles” might want to find results for “spaghetti”, “chow mein”, or “pad thai”. Synonyms can also help with typos, especially on mobile and small keyboards. In Atlas Search, you can define collections of synonyms for a search index via the API. Synonyms can be explicit (one-way) or equivalent (two-way). Explicit synonyms are good for defining relationships between terms that are subsets of each other, like the noodle example above: “spaghetti”, “chow mein”, and “pad thai” are all explicit synonyms for “noodles”, but not each other (you don’t want results for “chow mein” in a search for “spaghetti”). Equivalent synonyms are often used for terms that have regional variations or are otherwise interchangeable both ways, like soda and pop, or Kleenex and tissues. What's next for Atlas Search Developers are increasingly turning to full-text search to make content more discoverable and relevant for application end users. With Atlas Search, we hope to not only make building full-text search easier, but also more powerful and expressive. Join our community to ask questions and find out what other developers are building with Atlas Search and let us know what you think we should build next in our feedback forums .
Introducing Serverless Instances on MongoDB Atlas, Now Available in Preview
Since we first launched MongoDB Atlas in June 2016, we’ve been working towards building a cloud database that not only delivers a first-class developer experience, but also simply just works: no setup, tuning, or maintenance required. Over the years, this has led to features like auto-scaling and click-to-create index suggestions , along with numerous optimizations to our automation engine. We’re excited to announce that we’re one more step closer to realizing this vision with the introduction of serverless databases on MongoDB Atlas . Think less about your database, and more about your data Serverless computing and NoOps have emerged as popular trends in modern application development. Cloud functions are commonly used to power business logic in applications, and many teams rely on completely automated IT operations. The appeal of serverless technology is hard to deny: elastic scaling eliminates the need for upfront resource provisioning and ongoing maintenance, and consumption-based pricing means paying only for resources that are used. It abstracts and automates away many of the lower-level infrastructure decisions that developers don’t want to have to learn or manage so they can focus on building differentiated features. When it comes to databases, compute and storage resources have traditionally been tightly coupled. Applying a serverless model to databases means decoupling them and changing the way engineering teams think about infrastructure. Rather than asking a developer to predict an application’s future workload patterns, break them down into individual resource requirements, and then map them to arbitrary units of database instance sizes, serverless databases offer a much simpler experience: define where your data lives, and get a database endpoint you can use. This not only streamlines the database deployment process, it also eliminates the need to monitor and adjust capacity on an ongoing basis. Developers are free to focus on thinking about their data rather than their databases, and leave the lower-level infrastructure decisions to intelligent, behind-the-scenes automation. Serverless instances on MongoDB Atlas All customers now have the ability to create a serverless database on MongoDB Atlas with the introduction of serverless instances , announced at MongoDB.live 2021 . It’s incredibly easy to get started: simply choose a cloud region and you’ll receive an on-demand database endpoint for your application. Serverless instances always run on the latest MongoDB version so you never have to worry about backwards compatibility or upgrades. You can view and manage them using the same UI and API as your existing database deployment on Atlas (i.e., clusters), and they come with end-to-end security, continuous uptime, metrics, alerts, and backups. Watch this demo of how to create a serverless instance on MongoDB Atlas This new deployment type will be available in preview, so it doesn’t yet support all of the features and capabilities available on clusters today. It’s ideal for infrequent or sparse workloads, or development and testing workloads in the cloud. If you’re running a high-throughput production workload, dedicated clusters are still the recommended deployment option. A hands-free database experience This is the first of many releases, and we have an ambitious roadmap ahead. We will continue to invest in making working with data ever more seamless and delightful for developers, from adding support for newer Atlas capabilities like full-text search and native visualizations , to even more intelligent automation and optimization. Create your own serverless instance on MongoDB Atlas. Try the Preview If you have feedback or questions, we’d love to hear them! Join our community forums to meet other MongoDB developers and see what they’re building with serverless instances. What's next for MongoDB Atlas Serverless instances are just one of many new additions to Atlas that we hope will make developers’ lives easier. Earlier this year, we added index removal suggestions to Performance Advisor and released a quick start for creating and managing clusters via the command line with the MongoDB CLI . We are also working on integrations with Vercel and Netlify , two popular serverless application platforms, to give developers an easy way to get started on MongoDB Atlas. What would make your development experience better on MongoDB Atlas? Share your feature requests in our feedback forums .
Streaming Time-Series Data Using Apache Kafka and MongoDB
There is one thing the world agrees on and it is the concept of time. Many applications are heavily time-based. Consider solar field power generation, stock trading, and health monitoring. These are just a few of the plethora of applications that produce and use data that contains a critical time component. In general, time-series data applications are heavy on inserts, rarely perform updates and are even more unlikely to delete the data. These applications generate a tremendous amount of data and need a robust data platform to effectively manage and query data. With MongoDB, you can easily: Pre-aggregate data using the MongoDB Query language and window functions Optimally store large amounts of time-series data with MongoDB time-series collections Archive data to cost effective storage using MongoDB Atlas Online Archive Apache Kafka is often used as an ingestion point for data due to its scalability. Through the use of the MongoDB Connector for Apache Kafka and the Apache Kafka Connect service, it is easy to transfer data between Kafka topics and MongoDB clusters. Starting in the 1.6 release of the MongoDB Connector for Apache Kafka, you can configure kafka topic data to be written directly into a time-series collection in MongoDB. This configuration happens in the sink. Configuring time series collections in the sink With MongoDB, applications do not need to create the database and collection before they start writing data. These objects are created automatically upon first arrival of data into MongoDB. However, a time-series collection type needs to be created first before you start writing data. To make it easy to ingest time-series data into MongoDB from Kafka, these collection options are exposed as sink parameters and the time-series collection is created by the connector if it doesn’t already exist . Some of the new parameters are defined as follows: timeseries.timefield Name of the top level field used for time. timeseries.expire.after.seconds This optional field determines the amount of time the data will be in MongoDB before being automatically deleted. Omitting this field means data will not be deleted automatically. If you are familiar with TTL indexes in MongoDB, setting this field provides a similar behavior. timeseries.timefield.auto.convert This optional field tells the connector to convert the data in the field into a BSON Date format. Supported formats include integer, long, and string. For a complete list of the new time-seris parameters check out the MongoDB Sink connector online documentation . When data is stored in time-series collections, MongoDB optimizes the storage and bucketization of your data behind the scenes. This saves a tremendous amount of storage space compared to the typical one document per data point data structure in regular collections. You can also explore the many new time and window functionalities within the MongoDB Query Language. For example, consider this sample document structure: { tx_time: 2021-06-30T15:47:31.000Z, _id: '60dc921372f0f39e2cd6cba5', company_name: 'SILKY CORNERSTONE LLC', price: 94.0999984741211, company_symbol: 'SCL' } You can use the new $setWindowFields pipeline to define the window of documents to perform an operation on then perform rankings, cumulative totals, and other analytics of complex time series data. For example, using the data generated in the tutorial, let’s determine the rolling average to the data as follows: db.StockDataTS.aggregate( [ { $match: {company_symbol: 'SCL'} }, { $setWindowFields: { partitionBy: '$company_name', sortBy: { 'tx_time': 1 }, output: { averagePrice: { $avg: "$price", window: { documents: [ "unbounded", "current" ] } } } } } ]) A sample of the result set is as follows: { tx_time: 2021-06-30T15:47:45.000Z, _id: '60dc922172f0f39e2cd6cbeb', company_name: 'SILKY CORNERSTONE LLC', price: 94.06999969482422, company_symbol: 'SCL', averagePrice: 94.1346669514974 }, { tx_time: 2021-06-30T15:47:47.000Z, _id: '60dc922372f0f39e2cd6cbf0', company_name: 'SILKY CORNERSTONE LLC', price: 94.1500015258789, company_symbol: 'SCL', averagePrice: 94.13562536239624 }, { tx_time: 2021-06-30T15:47:48.000Z, _id: '60dc922472f0f39e2cd6cbf5', company_name: 'SILKY CORNERSTONE LLC', price: 94.0999984741211, company_symbol: 'SCL', averagePrice: 94.13352966308594 } Notice the additional “averagePrice” field is now populated with a rolling average. For more information on time-series collection in MongoDB check out the online documentation . Migrating existing collections To convert an existing MongoDB collection to a time-series collection you can use the MongoDB Connector for Apache Kafka. Simply configure the source connection to your existing collection and configure the sink connector to write to a MongoDB time series collection by using the “timeseries.timefield” parameter. You can configure the source connector to copy existing data by setting the “copy.existing” parameter to true. This will create insert events for all existing documents in the source. Any documents that were inserted during the copying process will be inserted once the copying process has finished. While not always possible, it is recommended to pause writes to the source data while the copy process is running. To see when it finishes, you can view the logs for the message, “Finished copying existing data from the collection(s).”. For example, consider a source document that has this structure: { company_symbol: (STRING), company_name: (STRING), price: (DECIMAL), tx_time: (STRING) } For the initial release of MongoDB Time series collections, the field that represents the time is required to be stored as a Date. In our example, we are using a string to showcase the ability for the connector to automatically convert from a string to a Date. If you chose to perform the conversion outside of the connector you could use a Single Message Transform in Kafka Connect to convert the string into a Date at the Sink. However, certain SMTs like Timestampconverter require schemas to be defined for the data in the Kafka topic in order to work. This may add some complexity to the configuration. Instead of using an SMT you can automatically convert into Dates using the new timeseries.timefield.auto.convert, and timeseries.timefield.auto.convert.date.format options. Here is a sample source configuration that will copy all the existing data from the StockData collection then continue to push data changes to the stockdata.Stocks.StockData topic: {"name": "mongo-source-stockdata", "config": { "tasks.max":"1", "connector.class":"com.mongodb.kafka.connect.MongoSourceConnector", "key.converter":"org.apache.kafka.connect.storage.StringConverter", "value.converter":"org.apache.kafka.connect.json.JsonConverter", "publish.full.document.only": true, "connection.uri":(MONGODB SOURCE CONNECTION STRING), "topic.prefix":"stockdata", "database":"Stocks", "collection":"StockData", "copy.existing":"true" }} This is a sample configuration for the sink to write the data from the stockdata.Stocks.StockData topic to a MongoDB time series collection: {"name": "mongo-sink-stockdata", "config": { "connector.class":"com.mongodb.kafka.connect.MongoSinkConnector", "tasks.max":"1", "topics":"stockdata.Stocks.StockData", "connection.uri":(MONGODB SINK CONNECTION STRING), "database":"Stocks", "collection":"StockDataMigrate", "key.converter":"org.apache.kafka.connect.storage.StringConverter", "value.converter":"org.apache.kafka.connect.json.JsonConverter", "timeseries.timefield":"tx_time", "timeseries.timefield.auto.convert":"true", "timeseries.timefield.auto.convert.date.format":"yyyy-MM-dd'T'HH:mm:ss'Z'" }} In this sink example, the connector will convert the data in the “tx_time” field into a Date and parse it expecting the string format yyyy-MM-ddTHH:mm:ssZ (e.g. '2021-07-06T12:25:45Z') Note that in the initial version of time-series collections, only insert into a time-series collection is supported. Updating or deleting documents on the source will not propagate to the destination. Also, you can not use the MongoDB CDC Handler in this scenario because the handler uses ReplaceOne which is a type of update command. These are limitations of the initial release of time-series in MongoDB and may be irrelevant by the time you read this post. Check the online documentation for the latest information. The MongoDB Connector for Apache Kafka version 1.6 is available to download from GitHub . Look for it on the Confluent Hub later this week!
The new MongoDB Shell is GA!
The new MongoDB Shell (mongosh) is now GA and becomes the default shell for the MongoDB platform. Download it now and start using it right away. Like all software we build at MongoDB, a great user experience is a major consideration. Even when working with a command-line tool, we believe it is just as important. To deliver this great experience to our users, we have redesigned the MongoDB Shell from the ground up to provide a modern command-line experience with enhanced usability features and a powerful scripting environment. After 1 year in beta, with a lot of great feedback from users and customers, we are excited to announce the general availability of the MongoDB Shell , the best way to work with your data and with your MongoDB deployments from the command line. The new MongoDB Shell is compatible with MongoDB 4.0+, so you don’t have to wait to upgrade to MongoDB 5.0 to start using it. You can download it and try it out now ! Enhanced user experience To make queries and aggregations easier to write and results easier to read, the MongoDB Shell comes with syntax highlighting. Now, it’s much easier to distinguish fields, values, and data types, which helps avoid syntax errors. If an error still occurs, the shell points you to the problem and helps you understand how to fix it. To help you type your queries and commands faster, the new MongoDB Shell includes intelligent autocomplete: based on the version of MongoDB you are connected to, the shell can suggest autocomplete options for methods, commands, and even MQL expressions. And when you don’t remember the syntax for a command, you can quickly look it up directly from the shell. Advanced scripting environment The MongoDB Shell is a great scripting environment. It’s built on top of the Node.js REPL, which means you can use the entire Node.js API in your scripts. Not only that: in your scripts for the MongoDB Shell you can now use any modules from npm. In the video below, you can see how I used node-fetch to fetch some data from a REST API and store it in MongoDB. Of course, you can also load and run scripts from the filesystem: as with the legacy mongo shell, in mongosh you can keep using load and eval to execute your scripts. Extensibility and snippets One of the goals we set for ourselves when we started building a new MongoDB Shell, was to make it easy to extend. This way, as the MongoDB platform grows with new products and services, the shell can grow with it. We also wanted to give our users and customers the possibility to extend the shell with all the functionality they need to be productive with MongoDB. While that has somewhat always been possible by loading scripts at startup with an RC file , we decided to take it one step further. In mongosh, we allow you to install Snippets. Snippets are plugins that you can install and are automatically loaded into the shell. Snippets can use any Node.js API and npm packages, allowing you to support a wide variety of use cases. We maintain a repository with a few Snippets that offer some new, interesting functionality (e.g. a snippet to analyze the schema of a given collection ) but you are free to configure mongosh to use a registry of snippets of your choice. Snippets are currently an experimental feature of mongosh. We are curious to see how you use it and get feedback so we can take it in the right direction and help you customize the shell exactly the way you need it. What happens with the legacy Mongo shell? You might be wondering what will happen with the legacy mongo shell. We are not taking it away quite yet. However, starting with MongoDB 5.0 the legacy shell is deprecated, and we encourage you to switch to mongosh as your default shell. Get started with mongosh! The new MongoDB Shell is available in our download center . Install it, connect with a MongoDB cluster and start scripting! Learn more about the MongoDB Shell and how to use it in our online documentation . If you have feedback or if you would like to suggest new features please let us know through our feedback engine .
Visualize Blended Atlas and AWS S3 Data From Atlas Data Lake with MongoDB Charts
We’re excited to announce that MongoDB Charts supports Atlas Data Lake as a data source! You can now use Charts to easily visualize data stored across different Atlas databases and AWS S3 buckets. Thanks to the aggregating power of Atlas Data Lake’s federated query, creating charts and graphs from blended application and cloud object data is simpler than ever before. On the surface this powerful integration is as simple as adding your Atlas Data Lake as a data source within Charts. However, it unlocks a deeper level of analysis while eliminating the need for creating an Extract-Transform-Load (ETL) process across your Atlas and S3 data. The integration provides the ability to visualize data from the following combination of sources without writing any code: Data from many Atlas databases or clusters, including multi-cloud clusters Cloud storage data from AWS S3 Blended Atlas and cloud storage (AWS S3) data Scenario: Finding insights from aggregated customer profile and contract data Let’s add a real world scenario of how this can enhance the analytics you derive from your data. While doing so, we will walk through the steps of setting up your Atlas Data Lake, adding it as a data source to Charts, and getting the most of your data with Charts’ powerful visualization capabilities. For context, let’s imagine we’re an analyst at a telecom company and we have contract data that is stored in MongoDB Atlas in different clusters and databases for each country we operate in - United States and Canada. Second, we have offloaded data from our Customer Relationship Management (CRM) tool as a parquet file into an AWS S3 bucket. All three datasets share a common “customerID” field. Configure Atlas Data Lake Because both “contracts” collections (or datasets) in MongoDB Atlas share the same fields, I simply mapped both into a single collection within the data lake. I mapped the customer profiles dataset into its own collection, since it only shares the “customerID” field. However, now that it’s in the same data lake, I will easily be able to join it to my contract data with a $lookup in my Charts aggregation pipeline or with a Lookup Field in the chart builder. (A $lookup in the MongoDB Query API is equivalent to a join in SQL.) Configure Charts data source I want to find insights from all contracts, both US and Canada in this scenario. Once I have created a single Atlas Data Lake collection (DL_contracts.allcontracts) from the two separate databases, I then need to add it as a data source in Charts. Simply click on “add data source” within Charts and add your data lake, and then choose the collections we want to use in the next step. For completeness I also added the two Atlas collections (US and Canada contracts) as data sources in Charts by following the same steps. Visualize data across multiple Atlas databases With Atlas Data Lake’s federated query capability, which effectively performs a union of data, I am able to build a column chart that shows the amount of all US and CA contracts in a single chart without writing any code. As you can see below, the chart shows both US and CA columns when connected to the data lake collection. When the data source is switched directly to either Atlas database, it only shows data for that respective database, or country in this example. Visualize blended data from Atlas and an AWS S3 bucket Lastly, let’s take our insights to the next level by visualizing data from multiple Atlas databases and a parquet file that’s stored in an AWS S3 bucket. Adding customer profile data that I offloaded from my CRM tool into S3 will enable me to find more robust insights. I could also visualize the data from the parquet file alone by connecting to that data lake collection. Since the contract data and customer profile data are in different collections within my Atlas Data Lake, I created a $lookup in the aggregation pipeline of the Charts data source. I then created a table chart from three different data sources with conditional formatting to quickly identify high value customers. The columns with blue boxes include contract data from both Atlas clusters, while the columns with orange boxes include customer profile data from a parquet file via AWS S3 bucket. Note, I could also aggregate the data in Atlas Data Lake and use $out to create a new collection of the data , and then connect Charts to the new collection as a data source. For the purposes of this blog, I wanted to highlight Charts-specific aggregation capabilities. We hope that you’re excited about the ability to easily visualize multiple data sources, from multiple Atlas databases to AWS S3 buckets in one place! Remember, if you haven’t used Charts before, you can get started for free by signing up for MongoDB Cloud , deploying an Atlas cluster and activating Charts. Try MongoDB Atlas for free today!
Distinguish Data, Get Insights Faster with Conditional Formatting in Charts
The latest release of MongoDB Charts adds Conditional Formatting; an exciting new feature that enables chart authors to highlight important changes in their chart data, based on a set of rules that they define. Conditional Formatting rules can be applied to table charts and number charts . Why use Conditional Formatting? For table charts, the data is densely packed into the visualisation using rows and columns. This is great for comparing many values simultaneously, but as the density increases it may become more difficult to find and focus on the data that matters. Many authors use Number charts to track key individual metrics within their data. While the number itself can be useful, sometimes it isn’t enough to convey other necessary information for its context – for instance, is a high number good or bad? Conditional Formatting can aid users in understanding the data by applying different styles based on rules to highlight what is important, and to provide them with more context. See Conditional Formatting in Action with Formula 1 Data Formula 1 motorsport is what I like to refer to as the “sport of nerds”, because analyzing and understanding huge amounts of data, and being able to quickly make a decision on that analysis can be the difference between winning and losing. So let’s see how Conditional Formatting can help with this task using data from the 2021 FIA Formula 1 World Championship. Single Color Conditional Formatting Let’s start off with something simple. Below is a table showing the 2021 Drivers Championship after three rounds. A driver’s position in the championship is determined by the total number of points they have been awarded over successive rounds of the season. Let’s edit this chart and add Conditional Formatting to highlight the top three drivers in the championship with colors to represent 1st, 2nd, and 3rd place. Click on the Customize tab , and then click on the Conditional Formatting menu to expand the accordion. As you can see we haven’t yet defined any rules, so let’s add a new rule by clicking the + Add button. A drawer will open up from the left hand side of the screen displaying the Add Format Rules view. A conditional formatting rule must have at least one condition, and all conditions must match in order for the rule to be applied. Let’s highlight the row of the driver currently in 1st place by adding a single color rule with one condition. Since this rule will be determined by the driver’s current position in the championship, we need to add a condition to act on this data. We can target this field by selecting Pos from the Applies to select control. Now that we know what field we are targeting, we must next choose an operator to use for the comparison. Since we are only interested in data that matches a specific value, we select the Equal To numeric operator. Next we must provide an input for the operand to be compared to. For this rule we are only interested in highlighting the driver that is in first place in the championship, so we enter a value of 1 into the Input . You can think of this condition as saying; “is the value of the field Pos equal to the value of 1?” If it is, then apply the styling of this rule, otherwise do not. Finally we choose what styling changes should be applied by choosing from the options under Styling . In this example, we want to highlight the background color of the cell in a gold color to signify 1st place, and we will also apply a bold font weight to the text to make it more prominent. Additionally we also would like for these styles to be applied to the entire row, and not just the cell that the condition is applicable to, so we will check the Format entire row option. And that’s it! Once we save the rule, you’ll notice that the table re-renders in the Chart Builder Preview to show that the data is being evaluated correctly and the Conditional Formatting rule is applied. We then simply rinse and repeat this process to add additional rules to highlight the drivers in 2nd and 3rd place, resulting in the following output: Color Scale Conditional Formatting When comparing tabular data, sometimes it is desirable to use color to show where each value lies relative to other values in the column. The table below shows the race results for the third round of the 2021 FIA Formula 1 World Championship. Each row displays the final result for each driver taking part in the race. Let’s compare the Average Speed of each driver’s Fastest Lap using a Color Scale Conditional Formatting rule. Navigate to the Add Format Rule screen in the same way by going to “Customize > Conditional Formatting > + Add” , but this time select the Color Scale radio option. Note: Conditional Formatting Color Scale rules can only have one condition, and this condition can only be applied to fields in a Table Chart encoded as Value columns. Select the Fastest Lap Average Speed as the target for the condition. You’ll notice that unlike the discrete Single Color rules, there are no other settings to configure for the condition. This is because a Color Scale will compare the values across the documents in sort order, and will determine a background color to apply to the cell based on the rank of the value within the range. Since we are interested in finding the highest Average Speed across each driver’s fastest lap, we will select a sequential color scale, where higher values are colored green, and lower values are colored in white. Save the rule to see the changes applied. As you can see, for the third round of the 2021 FIA Formula 1 World Championship, the fastest lap average speed was set by Valtteri Bottas at a blistering speed of 209.74 km/h (130.32 mph)! And there we have it. I hope this brief introduction to Conditional Formatting has highlighted (pun intended) the capabilities of this exciting new feature! In this post we’ve only scratched the surface of what’s possible though, Conditional Formatting has many many more powerful operators than what we have demonstrated here, including matching values by range, regular expression and even ranks. Why not take it for a test drive yourself to see what is possible? If you haven’t tried Charts yet, it’s quick, easy and free to get started. Simply sign up for MongoDB Cloud , deploy a free Atlas cluster and click Charts in the top navigation bar. You can also ask questions on the MongoDB Developer Community Forums , or suggest new or improved features using the MongoDB Feedback Engine .