Atlas

114 results

Fine-Tune Relevance in MongoDB Atlas Search with Function Scoring and Synonyms

MongoDB Atlas Search is an embedded full-text search solution in MongoDB Atlas that gives developers a seamless and scalable experience for building fast, relevance-based application features. We announced its general availability last year at MongoDB.live 2020 and over the past year we’ve introduced many new features, including a visual index builder, search query tester, custom analyzers , and wildcard path queries . This year at MongoDB.live 2021 , we’re excited to highlight two new capabilities that help developers tune the relevance of search results. See how easy it is to get started with MongoDB Atlas Search in this demo video by Marcus Eagan, Senior Product Manager for Atlas Search. Building relevance into search results Understanding the behavior of your users is essential when thinking about search result relevance. People don’t always tell you what they want, and they sometimes use words or phrases that don’t match your content exactly. To cover these scenarios, you can use full-text search features like function scoring and synonyms. Influence search rankings with function scoring There are often multiple factors that influence how search results should be ranked. For example, let’s say you have a restaurant finder application. The explicit inputs are things like the user’s location and what they’re searching for, but what’s implied is that they likely want to see highly rated restaurants or ones with more reviews. What’s Cooking: a sample restaurant finder application using MongoDB Atlas Search Function scoring allows you to influence the order of results returned by manipulating the score of each result. In Atlas Search, that means you can use a numeric field in a document and apply a mathematical expression to it. For example, you might want to increase the score of restaurants that are sponsored or have higher star ratings. This can easily be accomplished within the same search query by simply adding the function option to the score parameter of your query. Learn more about how to use function scores in our developer tutorial . Show results for more search queries with synonyms Synonyms are often used to define terms that are semantically similar to each other to improve search results. For example, someone searching for “noodles” might want to find results for “spaghetti”, “chow mein”, or “pad thai”. Synonyms can also help with typos, especially on mobile and small keyboards. In Atlas Search, you can define collections of synonyms for a search index via the API. Synonyms can be explicit (one-way) or equivalent (two-way). Explicit synonyms are good for defining relationships between terms that are subsets of each other, like the noodle example above: “spaghetti”, “chow mein”, and “pad thai” are all explicit synonyms for “noodles”, but not each other (you don’t want results for “chow mein” in a search for “spaghetti”). Equivalent synonyms are often used for terms that have regional variations or are otherwise interchangeable both ways, like soda and pop, or Kleenex and tissues. What's next for Atlas Search Developers are increasingly turning to full-text search to make content more discoverable and relevant for application end users. With Atlas Search, we hope to not only make building full-text search easier, but also more powerful and expressive. Join our community to ask questions and find out what other developers are building with Atlas Search and let us know what you think we should build next in our feedback forums .

July 13, 2021

Launched Today: MongoDB 5.0, Serverless Atlas, and the Evolution of our Application Data Platform

Today we welcome you to our annual MongoDB .Live developer conference. Through our keynote and conference sessions we'll show you all the improvements, new features, and exciting things we've been working on since last year’s conference. What I want to do in this blog post is provide you with a summary of what we are announcing, and resources to help you learn more. While it's easy to focus on what we are announcing at this year's event, we actually started out on this journey 12 years ago by releasing the world’s most intuitive and productive database technology to develop with — MongoDB. And we believe the applications of the NEXT 10 YEARS will be built on data architectures that continue to optimize for the developer experience, allowing teams like yours to innovate at speed and scale. So how are we building on this vision? Today I am incredibly proud to announce three big things: The General Availability (GA) of MongoDB 5.0, the latest generation of our core database. It includes native support for time series workloads, new ways to future-proof your applications, multi-cloud privacy controls, along with a host of other improvements and new features. The preview release of serverless instances on MongoDB Atlas, which makes it even easier for development teams who don’t want to think about capacity management at all to get the database resources they need quickly and efficiently. Major enhancements to Atlas Data Lake, Atlas Search, and Realm Sync, which allow engineering teams to reduce architectural complexity and get more value out of their data by leveraging a unified application data platform. MongoDB 5.0 GA MongoDB 5.0 is the latest generation of the database most wanted by developers . Our new release makes it even easier to support a broader range of workloads, introduces new ways of future-proofing your apps, and further enhances privacy and security. This major jump in version number from MongoDB 4.4 – our prior GA version – to 5.0 reflects a new era for MongoDB's release cadence: We want to get new features and improvements into your hands faster. Starting with MongoDB 5.0, we will be publishing new Rapid Releases every quarter, which will roll up into Major Releases once a year for those of you that want to maintain the existing annual upgrade cadence. You can learn more about the new MongoDB release cadence from our blog post published last October. Digging into MongoDB 5.0, here is what’s new and improved: Native Time Series Designed for IoT and financial analytics, our new time series collections, clustered indexing, and window functions make it easier, faster, and lower cost to build and run time series applications, and to enrich your enterprise data with time series measurements. MongoDB automatically optimizes your schema for high storage efficiency, low latency queries, and real-time analytics against temporal data. Running your time series applications on our application data platform eliminates the time and the complexity of having to stitch together multiple technologies yourself. You can manage the entire time series data lifecycle in MongoDB – from ingestion, storage, querying, real-time analysis, and visualization through to online archiving or automatic expiration as data ages. Time series collections can sit right alongside regular collections in your MongoDB database, making it really easy to combine time series data with your enterprise data within a single versatile, flexible database – using a single query API to power almost any class of workload. Our new time-series collections blog post gives you everything you need to get started. Future-proof with the Versioned API and Live Resharding Starting with MongoDB 5.0, the Versioned API future-proofs your applications. You can fearlessly upgrade to the latest MongoDB releases without the risk of introducing backward-breaking changes that require application-side rework. Using the new versioned API decouples your app lifecycle from the database lifecycle, so you only need to update your application when you want to introduce new functionality, not when you upgrade the database. Future-proofing doesn’t end with the Versioned API. MongoDB 5.0 also introduces Live Resharding which allows you to easily change the shard key for your collections on demand – with no database downtime – as your workload grows and evolves. The way I like to think about this is that we’ve extended the flexibility the document model has always given you down to how you distribute your data. So as things change, MongoDB adapts without expensive schema or sharding migrations. Next-Gen Privacy & Security MongoDB’s unique Client-Side Field Level Encryption now extends some of the strongest data privacy controls available anywhere to multi-cloud databases. And with the ability in 5.0 to reconfigure your audit log filters and rotate x509 certificates without downtime you maintain a strict security posture with no interruption to your applications. Run MongoDB 5.0 Anywhere MongoDB 5.0 is available today as a fully-managed service in Atlas . You can of course also download and run MongoDB 5.0 on your own infrastructure, either with the community edition of MongoDB, or with MongoDB Enterprise Advanced . The Enterprise Advanced offering provides sophisticated operational tooling via Ops Manager, advanced security controls, proactive 24x7 support, and more. MongoDB Ops Manager 5.0 enhancements include: Support for the automation, monitoring, and backup/restore of MongoDB 5.0 deployments. Improved load performance with parallelized client-side restores. A quick start experience for deploying MongoDB in Kubernetes with Ops Manager. And lastly, a guided Atlas migration experience that walks users through provisioning a migration host to push data from their existing environment into the fully managed Atlas cloud service. You can learn more about MongoDB 5.0 from our What’s New guide . New to MongoDB Atlas — Serverless Instances (Preview) We want developers to be able to build MongoDB applications without having to think about database infrastructure or capacity management. With serverless instances on MongoDB Atlas, now available in Preview, you can automatically get the database resources you need based on your workload demand. It’s really simple: the only decision you need to make is the cloud region hosting your data. After that, you’ll get an on-demand database endpoint that dynamically adapts to your application traffic. Serverless instances will support the latest MongoDB 5.0 GA release, Versioned API, and upcoming Rapid Releases so you never have to worry about backwards compatibility or upgrades. Pay only for reads and writes your application performs and the storage resources you use (up to 1TB of storage in preview) and leave capacity management to MongoDB Atlas’s best-in-class automation. We invite you to try it out today with a new or existing Atlas account. And the Preview release is just the beginning – we will be working with partners such as Vercel and Netlify to deliver an integrated serverless development experience in the coming months. In the longer term, we will continue to evolve our cloud-native backend architecture to abstract and automate even more infrastructure decisions and optimizations to deliver the best database experience on the market. The New MongoDB Shell GA The new MongoDB Shell has been redesigned from the ground up to provide a modern command-line experience with enhanced usability features and powerful scripting environment. It makes it even easier for users to interact and manage their MongoDB data platform, from running simple queries to scripting admin operations. A great user experience, even on a command-line tool, should always be a major consideration. With the new MongoDB Shell we have introduced syntax highlighting, intelligent auto-complete, contextual help and useful error messages creating an intuitive, interactive experience for MongoDB users. Check out this blog post for more information. MongoDB Charts and Atlas Data Lake: Better Together MongoDB Charts intuitive UI and ability to quickly create & share charts and graphs of JSON data is now integrated with Atlas Data Lake . You can now easily visualize JSON data stored in Amazon AWS S3 without any data movement, duplication or transformation. Furthermore, you can run Atlas Data Lake’s federated query to blend data across multiple Atlas databases and AWS S3, and visualize the results with Charts. By adding Atlas Data Lake as a data source in Charts, you can discover deeper, more meaningful insights in real time. Check out this blog post for more information. Atlas Search — More Relevance Features It’s incredibly important for modern applications to deliver fast and relevant search functionality: it powers discoverability and personalization of content, which in turn drives user engagement and retention. Atlas Search , which delivers powerful full-text search functionality without the need for a separate search engine, has several new capabilities for building rich end user experiences. We’ve recently added support for function scoring, which allows teams to apply mathematical formulas on fields within documents to influence their relevance, such as popularity or distance — e.g. closer restaurants with more or better reviews will show up higher in a list of results. In addition, you can now define collections of synonyms for a particular search index. By associating semantically equivalent terms with each other, you can respond to a wider range of user-initiated queries in your applications. Realm Realm lets you have simple, powerful local persistence on mobile phones, tablets and IoT devices like Raspberry Pi. The Realm SDKs provide a set of APIs that let developers store and interact with native objects directly, reducing the amount of code required as there is no need for ORMs or learning cryptic database syntax. In addition, we made MongoDB Realm Sync generally available earlier this year, making it easy to synchronize data between local storage on your devices and MongoDB Atlas on the backend. No need to worry about networking code or dealing with conflict resolution as we handle all of that for you. Today, we’re excited to announce support for Unity. You can now use Realm to store your game data, like scores and player state, and sync it automatically across devices. Realm's support for Unity is now Generally Available and ready for production workloads. We're also investing in support for more cross-platform frameworks — the Kotlin Multiplatform and Flutter/Dart SDKs are now both available in Alpha. And finally, the team is working towards Realm Flexible Sync, a new way to synchronize data with more granular control. Flexible Sync will allow you to — Build applications that respond dynamically to user's needs. Let your end users decide what data they need, and when. Use more precise permissions that can adapt over time. Check out this dedicated blog on our upcoming plans for Flexible Sync to learn more. Getting Started With everything we announced today, you can imagine it was a packed keynote! And there is so much more that we didn’t cover. You can get all of the highlights from our new announcements page where you will also find all the resources you need to get started.

July 13, 2021

Visualize Blended Atlas and AWS S3 Data From Atlas Data Lake with MongoDB Charts

We’re excited to announce that MongoDB Charts supports Atlas Data Lake as a data source! You can now use Charts to easily visualize data stored across different Atlas databases and AWS S3 buckets. Thanks to the aggregating power of Atlas Data Lake’s federated query, creating charts and graphs from blended application and cloud object data is simpler than ever before. On the surface this powerful integration is as simple as adding your Atlas Data Lake as a data source within Charts. However, it unlocks a deeper level of analysis while eliminating the need for creating an Extract-Transform-Load (ETL) process across your Atlas and S3 data. The integration provides the ability to visualize data from the following combination of sources without writing any code: Data from many Atlas databases or clusters, including multi-cloud clusters Cloud storage data from AWS S3 Blended Atlas and cloud storage (AWS S3) data Scenario: Finding insights from aggregated customer profile and contract data Let’s add a real world scenario of how this can enhance the analytics you derive from your data. While doing so, we will walk through the steps of setting up your Atlas Data Lake, adding it as a data source to Charts, and getting the most of your data with Charts’ powerful visualization capabilities. For context, let’s imagine we’re an analyst at a telecom company and we have contract data that is stored in MongoDB Atlas in different clusters and databases for each country we operate in - United States and Canada. Second, we have offloaded data from our Customer Relationship Management (CRM) tool as a parquet file into an AWS S3 bucket. All three datasets share a common “customerID” field. Configure Atlas Data Lake Because both “contracts” collections (or datasets) in MongoDB Atlas share the same fields, I simply mapped both into a single collection within the data lake. I mapped the customer profiles dataset into its own collection, since it only shares the “customerID” field. However, now that it’s in the same data lake, I will easily be able to join it to my contract data with a $lookup in my Charts aggregation pipeline or with a Lookup Field in the chart builder. (A $lookup in the MongoDB Query API is equivalent to a join in SQL.) Configure Charts data source I want to find insights from all contracts, both US and Canada in this scenario. Once I have created a single Atlas Data Lake collection (DL_contracts.allcontracts) from the two separate databases, I then need to add it as a data source in Charts. Simply click on “add data source” within Charts and add your data lake, and then choose the collections we want to use in the next step. For completeness I also added the two Atlas collections (US and Canada contracts) as data sources in Charts by following the same steps. Visualize data across multiple Atlas databases With Atlas Data Lake’s federated query capability, which effectively performs a union of data, I am able to build a column chart that shows the amount of all US and CA contracts in a single chart without writing any code. As you can see below, the chart shows both US and CA columns when connected to the data lake collection. When the data source is switched directly to either Atlas database, it only shows data for that respective database, or country in this example. Visualize blended data from Atlas and an AWS S3 bucket Lastly, let’s take our insights to the next level by visualizing data from multiple Atlas databases and a parquet file that’s stored in an AWS S3 bucket. Adding customer profile data that I offloaded from my CRM tool into S3 will enable me to find more robust insights. I could also visualize the data from the parquet file alone by connecting to that data lake collection. Since the contract data and customer profile data are in different collections within my Atlas Data Lake, I created a $lookup in the aggregation pipeline of the Charts data source. I then created a table chart from three different data sources with conditional formatting to quickly identify high value customers. The columns with blue boxes include contract data from both Atlas clusters, while the columns with orange boxes include customer profile data from a parquet file via AWS S3 bucket. Note, I could also aggregate the data in Atlas Data Lake and use $out to create a new collection of the data , and then connect Charts to the new collection as a data source. For the purposes of this blog, I wanted to highlight Charts-specific aggregation capabilities. We hope that you’re excited about the ability to easily visualize multiple data sources, from multiple Atlas databases to AWS S3 buckets in one place! Remember, if you haven’t used Charts before, you can get started for free by signing up for MongoDB Cloud , deploying an Atlas cluster and activating Charts. Try MongoDB Atlas for free today!

July 9, 2021

MongoDB Atlas Celebrates Five Years of Innovation in Data

Today we’re thrilled to celebrate the five-year anniversary of Atlas, MongoDB’s multi-cloud database platform. When we launched Atlas in 2016, we couldn’t have foreseen the impact it would have on both our customers and MongoDB as a company. MongoDB Atlas has allowed us to become an ever more trusted partner to our customers, playing a key role in their efforts to manage and mitigate risk. One of the insights that spurred the development of Atlas — that it should be easy to move your data into, out of, and between clouds — remains as groundbreaking today as it was five years ago. Thanks to our commitment to innovation, reliability, and security, Atlas’s customers include well-known companies such as Forbes , Toyota Material Handling , Pitney Bowes , and 7-Eleven . Sixty of the Fortune 100 and many of the world’s most innovative disruptors rely on Atlas to help them grow, become more efficient, gain insights from their data, and create superior customer experiences. Atlas has transformed MongoDB into a cloud-first company: Atlas’s revenue is growing at 73 percent a year and currently accounts for more than half of MongoDB’s revenue. We got a glimpse of the future in 2009 with the first production version of MongoDB. For years, developers struggled to build modern applications on top of decades-old relational databases. Using a JSON-based document model, MongoDB was exceptionally fast, scalable, flexible, and intuitive for developers. It was unusually proficient with both structured and unstructured data. The database’s very design pushed developers and engineers to think differently about how they worked with data. As our customers investigated new business models enabled by the cloud, we noticed two things about the way they were working with data. First, they were increasingly choosing to use MongoDB in the cloud rather than hosting it on premises. Second, our customers were gravitating toward managed services. If our customers were going to the cloud — and they were — we needed to be there too. Our goal was not only to become a cloud-native database, but also to provide developers with a superior platform so they could change the world with data — and employ all the cloud’s potential to do it. As a managed service, Atlas would free developers from the overhead of managing MongoDB themselves. By making data portable across the biggest public clouds, such as Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure, it would free data from vendor lock-in. Developers and engineering teams could match the right cloud to the right workload in a few clicks and a few minutes without writing or rewriting code. Since Atlas shipped in 2016, it has gained more than 25,000 customers. We’ve dramatically improved its functionality, turning it into an even more powerful tool to fulfill our mission of making data stunningly easy to use. At its 2016 launch, Atlas was available in four AWS regions. The next year, we introduced cross-region replication. Within a single cloud, a customer could now enable cross-region deployments for even better availability guarantees. We launched global clusters in 2018, which made it easier to place data closer to the user and enabled our customers’ applications to run faster all over the world. MongoDB Atlas is now available in about 80 regions across AWS, Microsoft Azure, and GCP. Atlas customers can switch clouds as business requirements change to take advantage of pricing changes and make the best use of each cloud’s capabilities. Our broad reach also helps our customers comply with increasingly complex data sovereignty and localization requirements. We’ve pushed Atlas well beyond the ability to serve larger numbers of regions. In 2019 we acquired Realm , which makes data accessible no matter where your user is or how lousy their connectivity may be. The MongoDB Realm Mobile Database enables simple, powerful persistence on mobile devices so apps can work offline as well as they do online. Two years later, we released MongoDB Realm Sync, making it even easier to keep data in sync across users and devices and connect to the Atlas database on the backend. We also released MongoDB Atlas Search, which enables developers to create rich, relevance-based search without moving their data into a separate search engine. That was accompanied by MongoDB Atlas Data Lake, enabling developers to use federated queries to analyze data across tiers. In 2019 we introduced client-side field-level encryption, an industry-leading approach to security. It’s relatively common to encrypt data at rest or in transit, but client-side field-level encryption encrypts data while it’s in use. There’s no additional code for developers to write and no significant impact on performance, and applications can still query data. Client-side field-level encryption enables our clients to use managed services in the cloud with more confidence, because even those who support the underlying cloud infrastructure cannot decrypt the data. It also makes it easier to comply with increasingly common “right to be forgotten” mandates in contemporary privacy legislation. A user can be forgotten simply by destroying the associated encryption key, making their data unreadable and irrecoverable. In 2020, our development teams accomplished what had been their mission since the inception of Atlas: multi-cloud clusters. With multi-cloud clusters, MongoDB Atlas goes well beyond its promise — fulfilled years earlier — to work equally well in any of the public clouds. Multi-cloud clusters enable a single cluster to be in multiple clouds simultaneously, making it trivial to move data between them. We have big plans for the next five years, and we’re already getting started. Soon we’ll preview serverless instances on MongoDB Atlas, making it even easier for development teams to get the capacity they need, when they need it. You choose the region that hosts your data and we’ll do the rest, with an on-demand database endpoint that dynamically adapts to your application traffic. We’re also making it easier to support a broader range of workloads, offering new ways to future-proof apps, and continuing to improve security and privacy capabilities. We’ll be making major enhancements to Atlas Data Lake, Atlas Search, and Realm Sync, all of which reduce architectural complexity and allow our customers to get more value from their data with a unified application data platform. And we’ll be doing it all on an accelerated cadence. Starting with MongoDB 5.0, we’ll publish new releases every quarter for those who want to be on the fast track, and then rolling those up into annual Major Releases for those who want to stay on the current cycle. We expect the next five years to be just as exciting and innovative as the past five — if not more. We can’t wait to see you there. For more on Atlas's Five Year Anniversary, check out this video

June 28, 2021

Deploy and Manage MongoDB Atlas from AWS CloudFormation

As a premier launch partner for the recent GA announcement of the AWS CloudFormation Public Registry , we’re delighted to share that you can now deploy and manage MongoDB Atlas directly from your AWS environment. Amazon and MongoDB have been pioneers in the cloud computing space, providing mission critical systems for over a decade. Before MongoDB Atlas was launched in June 2016, tens of thousands of customers were running MongoDB themselves on AWS EC2 instances, and many of them were originally spun up using the legacy MongoDB on the AWS Cloud: Quick Start Reference Deployment. This Quick Start was among the top five most popular guides for AWS and allows users to take advantage of AWS CloudFormation 's seamless automation and MongoDB’s flexible data model and expressive query API. In April 2021, we launched a new AWS Quick Start for MongoDB Atlas , which allows AWS customers to quickly and easily launch a basic MongoDB Atlas deployment from the AWS CLI or console. Now, with the availability of the MongoDB Atlas resource types on the CloudFormation Public Registry, customers have more flexibility over their deployment configurations to better meet their cloud workflows. Let’s walk through how it works. Setup your AWS account for MongoDB Atlas CloudFormation Support The first step is to sign up for MongoDB Atlas , if you haven’t done so already. Once you create your account, follow these steps: Skip the cluster deployment options Go to Billing and add a credit card to your account Create an organization-level MongoDB Atlas Programmatic API Key with an IP Access List entry. The key needs Organization Project Creator permissions. Next, open the AWS console in your browser and navigate to CloudFormation. On the left-side navigation, select the Public extensions option. From there you will be able to find the MongoDB Atlas resource types by selecting the “Resource Types” and “Third Party” options. For each of the MongoDB::Atlas resource types, click “Activate”, and then follow on screen prompts to complete the process. Once you have activated the MongoDB Atlas resources in a region, you’re ready to launch apps with MongoDB Atlas directly from your AWS control plane. Build apps faster with Cloud Automation Context switching is a hassle for developers. Launching and deploying application stacks with MongoDB Atlas directly from the AWS console is now more seamless than ever. Whether you use the AWS Quick Start deployment guide as a template or create your own MongoDB Atlas CloudFormation templates, you can leverage the latest in cloud automation to reduce the pain of infrastructure provisioning and management. Try out the new MongoDB Atlas CloudFormation Resources today, and stay tuned for an in depth look at building apps with AWS Lambda and SAM CLI in an upcoming DevHub article!

June 21, 2021

Use x509 certificate-based authentication with MongoDB and Apache Kafka

Kafka has emerged as a popular event streaming platform. The inherent "pub/sub" model can be viewed as a method for moving data between systems. As such, MongoDB offers a Kafka connector , enabling Kafka topics to be copied into a MongoDB cluster (the sink). Similarly, the connector enables data movement from a MongoDB cluster (the source) into Kafka topics. To access data securely, certificate-based X.509 authentication is a natural choice for server-to-server authentication scenarios with Kafka and MongoDB. Certificates avoid having to store or manage usernames and passwords when used with database connection strings. For example, such user credentials could be inadvertently exposed if "hard-coded" in configuration files or other uses. An X.509 certificate is a structured, binary record. This record consists of several key and value pairs. X.509 certificates use the widely accepted international X.509 public key infrastructure (PKI) standard. The use of certificates prevents user credential exposure. Authentication requests with certificates verifies that any public key presented by a client or another member of the cluster belongs to that client or member. The X.509 certificate method for authentication is more secure than conventional password-based certification because each server machine needs their own dedicated key to participate in the cluster. For use with secure TLS/SSL connections, MongoDB supports X.509 certificate authentication allowing clients to use public key infrastructure in lieu of SCRAM (username and password). The certificate encodes two very important pieces of information: the server's public key and a digital signature that can be used to confirm the certificate's authenticity. Additionally, the certificate will include metadata used by the Certificate Authority to track the certificate and provide guidelines on how the public key can be used. Using the server's public key, the client and server are able to negotiate a shared symmetric key securely, which can be used to secure communications. Users can either generate their own certificates and keys (self-managed) or use the Atlas PKI. In either case, first a project-specific CA private and public key is generated, and then a per-user private key and signed X.509 identity certificate is created. If using self-managed X.509 infrastructure , you'll need to upload your CA public key certificate into your Atlas project. If using Atlas-managed X.509 infrastructure, you'll need to download the project private key and provide that to your Kafka Connect service. This signed certificate is then pushed to each server member in your Atlas cluster. The below diagram shows the deployment of a standard 3 node replica set and client using x.509 authentication: In non-production environments, the basic SCRAM authentication method may be most suitable. However, for production environments or server-server scenarios such as a Kafka-MongoDB integration, X.509 authentication is the recommended mechanism. To use X.509 certification for server-server authentication, first confirm that you are able to authenticate to an Atlas cluster using X.509 certificates. Then follow the steps below. Prerequisites: Openssl must be installed Project-level CA & user certificates created in PEM format If using Atlas-managed certificates, user-specific client certificate (see X.509 tab: https://docs.atlas.mongodb.com/security-add-mongodb-users/#database-user-authentication ) If using self-managed X.509 auth, you will need to create & upload your CA public key to Atlas (see https://docs.atlas.mongodb.com/security-self-managed-x509/ ), and have a user-specific client certificate ready Ensure that you have installed the MongoDB Kafka Connector and understand how to use it with Kafka Connect. Then follow these steps: Obtain the client user certificate from your system administrator (or from Atlas). In this example, the user certificate is stored in PEM file kafkaclient-X509-cert.pem and will be associated with the Atlas database user kafka-svc . Convert the PEM file to a password-protected PKCS12 formatted certificate by running this command: openssl pkcs12 -export -in kafkaclient-X509-cert.pem -out kafkaclient-X509-cert.p12 -password pass:mypassword Copy PKCS12 certificate ( kafkaclient-x509-cert.p12 ) to the server where Kafka Connect is running. Note the full path of the PKCS12 certificate location. Update the Kafka Connect configuration in the KAFKA_OPTS environment variable: export KAFKA_OPTS="-Djavax.net.ssl.keyStore=<path to kafkaclient-x509-cert.p12> -Djavax.net.ssl.keyStorePassword=mypassword -Djavax.net.ssl.keyStoreType=PKCS12" Restart Kafka Connect Update the MongoDB Connector configuration to use a connection URI with the following parameter options: Connection.uri: "mongodb+srv://<mongodb-host>/test?authSource=%24external&authMechanism=MONGODB-X509&subjectName=kafka-svc" Re-deploy the MongoDB connector using the Kafka Connect REST API, with the above configuration for the connection URI. Download the latest MongoDB Connector for Apache Kafka 1.5 from the Confluent Hub ! Read the MongoDB Connector for Apache Kafka documentation . Questions/Need help with the connector? Ask the Community .

May 5, 2021

New MongoDB Shell now supports Client-side Field-level Encryption

Last summer, we introduced mongosh , the new MongoDB Shell with an enhanced user experience and a powerful, Node.js-based scripting environment . Since then, we have been adding new functionality and APIs to close the gap with the legacy mongo shell, on the path to making it the default shell for MongoDB. In addition to the set of CRUD and other commands that we supported in the first release we recently added: Bulk operations Change Streams Sessions and Transactions Logging and profiling commands Replica set and Sharding configuration commands Plus some other minor things and utility commands here and there. This week, we released mongosh 0.8 with support for Client-side Field-level Encryption (FLE). Support for Client-side Field-level Encryption MongoDB Client-Side Field-level Encryption (FLE) allows developers to selectively encrypt individual fields of a document using the MongoDB drivers (and now with mongosh as well) on the client before it is sent to the server. This keeps data encrypted (but still queryable) while it is in-use in database memory, and protects it from the providers hosting the database, as well as from any user that has direct access to the database. Back in November, we announced that in addition to AWS’ KMS, Client-side FLE now supports key management systems in Azure and Google Cloud in beta. The most recent version of the MongoDB Shell makes it easy to test this functionality in a few easy steps: Create a free Atlas Cluster Install mongosh . Check out our documentation to set up your KMS in Azure or GCP. Start encrypting! To make it easier to get started with Client-Side FLE, here are two simple scripts that you can edit and copy-paste into mongosh: mongosh-fle-gcp-kms to set up Client-side FLE with Google Cloud and mongosh-fle-local-kms to use a local key. In the screenshot below, you can see a document that was encrypted on the client with automatic encryption before it was sent across the wire and inserted into MongoDB. Fields are in clear text in the shell but then are shown as encrypted when connecting with Compass to the same Atlas cluster. A Powerful Scripting Environment As mongosh is built on top of Node.js, it’s a great environment for scripting , no matter if it’s about checking the health status of your replica set or if you want to take a quick look at the data to make sure it’s coming in from your application as you are expecting. With modules from npm , the experience becomes much more rich and interactive. For example, if I want to look at the sample_mflix collection available in the Atlas sample datasets and check the distribution of thriller movies over the years, I can put together a simple script that includes running an aggregation and visually formatting the results with an open source library called babar This is just one of many ways you can extend the functionality of the MongoDB Shell by taking advantage of the great ecosystem of JavaScript libraries and modules that the community has built over the years and keeps on building every day. Start Scripting and Let Us Know How it's Working for You! As we added new functionality to the MongoDB Shell, we tried as much as possible to keep backwards compatibility with mongo, and we were mostly able to do that. In a limited number of cases, however, we took the opportunity to clean up the API and address some unexpected behaviors. Wondering what’s coming next in mongosh? We are working on adding support for load() and rc files to make it easy to load your scripts into the shell. If you find something that does not work as expected, please let us know! Simply create a bug in our JIRA project or reach out on Twitter .

February 22, 2021

MongoDB Rated an “Overall Leader” by Technology Analysts KuppingerCole

KuppingerCole, one of Europe’s leading technology analyst firms, has recognized MongoDB as an “overall leader” in a new report focused on enterprise databases in the cloud. In the report, KuppingerCole acknowledged MongoDB’s unique position as a database platform designed for the cloud age and well-suited to the applications that drive disruption, innovation, and competitive advantage. “Designed from scratch as a general-purpose database platform for the cloud age, MongoDB has grown into the world’s fastest-growing NoSQL ecosystem and one of the preferred engines for modern cloud-native applications,” wrote Alexei Balaganski, lead analyst for KuppingerCole. The report noted that, as the only overall leader in cloud databases that is not also a cloud services provider, MongoDB is uniquely committed to making sure data can move easily between multiple cloud providers (AWS, Microsoft Azure, and Google Cloud). In addition to the overall leadership accolade, KuppingerCole gave MongoDB its “Strong positive” designation – the highest possible rating – in seven important areas: security, functionality, interoperability, usability, deployment, innovativeness, and market position. Excellence in Product, Innovation, and Market Position MongoDB was one of only four database services to achieve a leadership designation in each of the three areas key to the final categorization as an overall leader: product leadership, innovation leadership, and market leadership. Product Leadership. Here, KuppingerCole singled out MongoDB Atlas as the core of MongoDB’s cloud-agnostic database product, offered as a managed cloud service across the three major cloud platforms (AWS, Azure, GCP). Balaganski also refers to MongoDB's popularity among developers, who have rated it the “Most Wanted” database four years running in StackOverflow’s annual survey . “MongoDB offers maximum deployment flexibility and an interface for developers so familiar that other vendors chose to implement the MongoDB protocol for their own databases,” writes Balaganski. Innovation Leadership. KuppingerCole defines leadership in innovation not so much as a constant flow of new features, but as a customer-oriented upgrade approach. The analyst firm wants to see backward compatibility, especially at the API level, and it wants to see new features that meet emerging customer requirements. Clearly, MongoDB delivers. KuppingerCole lauded MongoDB's history of listening to its customers and delivering new and relevant features. The result, the analyst firm said, is that MongoDB has been, “One of the most popular NoSQL databases for years.” KuppingerCole specifically pointed out Atlas Search, Atlas Data Lake, and mobile support. Market leadership. MongoDB was rated a leader in this category as well. While Oracle has, “Arguably the largest number of on-prem database customers,” it was far from assured that all of them would migrate to Oracle Cloud. KuppingerCole said MongoDB, “Dominates the NoSQL database market across all major cloud platforms.” MongoDB Atlas' Unique Advantages In its analysis of the “overall leader” cohort, KuppingerCole showed why it could confidently claim that MongoDB and the MongoDB Atlas cloud platform, “Provide a simple, universal alternative for developers that do not want to replicate the complexity of their legacy on-prem infrastructures in the cloud and would rather avoid a plethora of specialized database engines, opting instead for a single interface to manage, query, and analyze all of their data.” Two other prominent vendors, said KuppingerCole, focus on a database portfolio strategy, offering a purpose-built database for every conceivable need. But KuppingerCole said that approach runs the risk of locking data in silos, and complicates analytics across silos. The analyst firm said that management capabilities can vary between different database engines. It also noted that another leading vendor’s database engine was not well-suited to highly distributed modern application architectures. MongoDB addresses those challenges. Among its strengths, KuppingerCole noted that MongoDB provides a single platform for transactional, search, and analytics workloads. It also mentioned MongoDB’s multi-cloud availability in AWS, GCP, and Azure; its advanced security features, such as client-side field-level encryption; and comprehensive developer support with drivers, tools, and additional services. That makes it, as Balaganski writes, “A one-stop platform for developing modern highly scalable, and distributed cloud-native applications.” And, of course, an overall leader.

January 29, 2021

Add Interactivity to Your Embedded Analytics with Click Events

MongoDB Charts’ data visualizations can now become more interactive, so users and stakeholders can dive deeper into the insights they care more about. That’s possible with a new feature currently in beta with support for most Charts types: click events. A click event in the Charts embedding SDK is simply a notification that a user clicked on a chart. That click could be anything: They might have clicked on a bar in a bar chart, a chart’s legend, or even empty white space within the chart. Web developers can subscribe to these events and create different functionality depending on where the user clicked. Why might you want to enhance your app or embedded analytics workflow with click event data? Click-event data opens up a wide range of possibilities. Here are a couple of examples, inspired by various Charts users who’ve been telling us how they’d like to use click-event data. Open up another chart, based on a user clicking on data within a chart: A logistics company has a bar chart that shows pending orders at an aggregate level per region, and they want to see more detail on pending orders for a specific region. They create a click event handler in their application that opens up a new chart with pending orders per supplier, based on the region selected in the aggregate chart. Filtering the other charts on a dashboard when a series or data point on a single chart is clicked: A retail clothing company has a dashboard with various shopping cart information such as sales, orders processed, and returns, for their portfolio of products. The head of outerwear sales only wants to see data for the “outerwear” category of products, so they click on the “outerwear” series within a bar chart. The rest of the dashboard can adapt so that it shows only information relevant to outerwear. The example below is created from one of our sample data sets. We created two charts in a single app, tied to the sample movie data set that every Atlas user can access. On the left is a stacked bar chart with category level data that includes genre and decade. On the right is a table chart that shows each individual movie within a category. Clicking on a specific category in the bar chart updates the movies shown in the table chart. How can you get started with click events of embedded charts? If you haven’t yet used the embedding SDK for MongoDB Charts, you’ll want to familiarize yourself with the docs , consider watching this video tutorial , and access the SDK via the Charts GitHub repository . Regardless if you’re new to using the SDK or have experience with it, it’s important to note that you will need to use the @mongodb-js/charts-embed-dom@beta tagged version of the SDK to have access to the click events functionality while it’s in beta. There are two examples specifically for click events in the repository: click-events-basic and click-events-filtering . If you just want to explore and test out the functionality, you can play around with them in the following sandboxes using codesandbox.io: Click events basic sandbox Click events filtering sandbox Here’s a snapshot of the data surfaced in a click event that is available for developers to use in their apps. In this example I clicked on the yellow section of the top bar in the Movie Genres chart above. Note how it includes details of the click coordinates, the type and role of the clicked mark, and the corresponding chart data for the mark. { "chartId": "90a8fe84-dd27-4d53-a3fc-0e40392685dd", "event": { "type": "click", "altKey": false, "ctrlKey": false, "shiftKey": false, "metaKey": false, "offsetX": 383, "offsetY": 15, "clientX": 403, "clientY": 99, "pageX": 403, "pageY": 99, "screenX": 756, "screenY": 217 }, "data": { "y": { "label": "Genre", "value": "Drama" }, "x": { "label": "# Movies", "value": 3255 }, "color": { "label": "Decade", "value": "2010 - 2020", "lowerBound": 2010, "upperBound": 2020 } }, "target": { "type": "rect", "role": "mark", "fill": "#F0D175" }, "apiVersion": 1 } Whether you’re an avid user or new to MongoDB Charts, we hope you consider taking advantage of the new click event capability to increase the interactivity of Charts. It’s in beta because there is more functionality still to come. It has yet to be released for a few chart types: geospatial, table, top item, word cloud, and number charts. On that note, we’d love to hear your thoughts through the MongoDB Feedback Engine . If you haven’t tried Charts yet, you can get started for free by signing up for a MongoDB Atlas and deploying a free tier cluster.

January 20, 2021