New in Atlas Search: Improve Content Recommendations With “More Like This”
We’re proud to announce the release of More Like This, a key MongoDB Atlas Search feature that allows developers to easily build more relevant and engaging experiences for their end users. With the moreLikeThis operator, you can display documents that are similar to a result document. In this article, we’ll explain how it works and how you can get started using this new feature. Content recommendation done easily People who use travel booking apps, streaming services, and e-commerce websites are likely familiar with “Frequently Bought With,” “Similar Products,” or “You Might Also Enjoy” sections in their search experiences — in other words, content recommendation that guides them toward new or related products to buy, movies to stream, recipes to make, or news articles to read (among other things). Instead of building and tuning a recommendation engine to provide this functionality, developers can create engaging, browsable search experiences by defining a similarity threshold between documents to surface relevant documents. How it works Under the hood, the moreLikeThis search operator extracts the most representative terms from a reference document or documents and returns a set of similar documents. The representative terms are selected based on term frequency-inverse document frequency (TF-IDF), which is calculated by looking at a given term’s frequency in a given document multiplied by its frequency in the corpus. TF-IDF is calculated by looking at a term’s frequency multiplied by its frequency in the corpus. Atlas Search indexes term frequency by default, which means there is less up-front configuration required when compared with other search solutions. Additionally, developers have the ability to define what constitutes sufficient similarity for their use cases, with control over variables such as the number of query terms selected and the minimum and maximum document frequency thresholds. Use cases An example use case might look like this: An online bookstore wants to upsell users who have reached the checkout stage with similar books. On the checkout page, the user is served with a More Like This query result in the form of an “Other Books You Might Like” section that contains an array of book titles based on multiple fields in the document (e.g., title, publisher, genre, author). More Like This can be applied to use cases like ecommerce, content management systems, application search, or anywhere you want to share more relevant content with your users to drive deeper engagement. For more examples of how to configure More Like This, refer to our examples in the Docs . To learn how to get started with More Like This, refer to our documentation . For real-world Atlas Search implementation examples, go to our Developer Center .
5 Steps to Replacing Elasticsearch and Solr with Atlas Search
What do a global auto manufacturer, multinational media and entertainment company, and a challenger bank have in common? They have all made the switch from Elasticsearch to MongoDB Atlas Search to simplify their technology stack and ship application search faster. But what problems were they solving and how did they migrate? We have a new 5-step guide that takes you through why they switched, and how they did it. The need for application search Type almost anything into a search bar on sites like Google, Amazon, and Netflix and you are instantly presented with relevant results. Whether you make a typo or enter a partial search term, the search engine figures out what you are looking for. Results are returned conveniently sorted by relevance and are easy to navigate with features like highlighting, filters, and counts. Everyone now expects these same fast and intuitive search experiences in every application they use, whether at home or at work. However, creating these experiences is hard with the burden falling onto developers and ops teams who have to build and run the underlying systems. The pain of building application search MongoDB has always focused on accelerating and simplifying how developers build with data for any class of application. From our very earliest MongoDB releases, we saw developers needing to expose the application data stored in their database to search and information discovery. For simple use cases – where it was enough to just match text in a field – developers were able to use the basic text search operators and index built into the MongoDB database. However these lacked the much more sophisticated speed and relevance tuning features offered by dedicated search engines, typically built on top of Apache Lucene . As a result many developers ended up bolting on an external search engine such as Elasticsearch or Apache Solr to their database. Elasticsearch and Solr were (and remain) popular and proven. However as Figure 1 shows, they introduced a huge amount of complexity to the application stack, reducing developer velocity while driving up risk, complexity, and cost. Figure 1: The pain of bolting on a search engine to your database Working with the MongoDB community, our product designers and engineers ideated on ways to make building application search easier for developers – without compromising on the key features they needed. The result is MongoDB Atlas Search . What is Atlas Search and why switch to it? Atlas Search embeds a fully-managed Apache Lucene search index directly alongside the database and automatically synchronizes data between them. By integrating the database, search engine, and sync pipeline into a single, fully-managed platform you get to compress three systems into one and simplify your technology stack. Engineering teams and application owners have reported improved development velocity of 30% to 50% after adopting Atlas Search. This is because they get to: Eliminate the synchronization tax. Data is automatically and dynamically synced from the Atlas database to the Atlas Search indexes. They avoid having to stand up and manage their own sync mechanism, write custom transformation logic, or remap search indexes as their database schema evolves. They escape the 10% of engineering cycles typically lost to manually recovering sync failures, investing that time to innovate for their users instead. ( 1 ) Ship new features faster. They work with a single, unified API across both database and search operations, simplifying query development. No more context switching between multiple query languages, and with a single driver, build dependencies are streamlined so they release faster. They can test queries and preview results with interactive tools to fine-tune performance and scoring before deploying them directly into application code. Remove operational heavy-lifting. The fully-managed Atlas platform automates provisioning, replication, patching, upgrades, scaling, security, and disaster recovery while providing deep performance visibility into both database and search. By working with a single system, they avoid an exponential increase in the number of system components they need to design, test, secure, monitor, and maintain. Figure 2: Dramatic architectual simplification with integrated database, sync, and search in MongoDB Atlas 5 steps to make the switch to Atlas Search The benefits Atlas Search provides has led engineering teams across all industry sectors and geographies to make the switch from bolt-on search engines. Through the experiences gained by working with these teams, we have put together a repeatable 5-step methodology to replacing Elasticsearch and Solr. The guide steps you through how to: Qualify target workloads for Atlas Search. Migrate your indexes to Atlas Search. Migrate your queries to Atlas Search. Validate and relevance-tune your Atlas Search queries and indexes. Size and deploy your Atlas Search infrastructure. Figure 3: 5-step methodology to replacing Elasticsearch and Solr with Atlas Search The guide wraps up with examples of customers that have made the switch and provides guidance on how to get started with Atlas Search. What's next? You can get started today by downloading the 5-step guide to replacing Elasticsearch and Solr with Atlas Search . The 5-step guide is designed to help you plan and execute your migration project. MongoDB's Professional Services team is also available to you as a trusted delivery partner. We can help you through any of the steps in the methodology or throughout your entire journey to Atlas Search. If you want to dig deeper into Atlas Search, spin it up at no-cost on the Atlas Free Tier . You can follow along with reference materials and tutorials in the Atlas Search documentation using our sample data sets, or load your own data for experimentation within your own sandbox. Welcome to a world where application search is, at last, simplified! Download the 5-step Guide Now! 1. Based on interviews with engineering teams that have replaced bolt on search engines and the associated sync mechanism.
100x Faster Facets and Counts with MongoDB Atlas Search: Public Preview
Today we’ve released one of the most powerful features of Atlas Search in public preview, and ready for your evaluation: lightning fast facets and counts over large data sets. Faceted search allows users to filter and quickly navigate search results by categories and see the total number of results per category for at-a-glance statistics. With the new facet operator , facet and count operations are pushed down into Atlas Search’s embedded Lucene index and processed locally – taking advantage of 20+ years of Lucene optimizations – before returning the faceted result set back to the application. What this means is that now facet-heavy workloads such as ecommerce product catalogs, content libraries, and counts run up to 100x faster . The power of facets and counts in full-text search Faceting is a popular search and analytics capability that allows an application to group information into related categories by applying filters to query results. Users can narrow their search results by simply selecting a facet value as a filter criteria. They can intuitively explore complex data sets, providing fast and convenient navigation to quickly drill into the data that is of most interest. A common use of faceting is navigating product catalogs. With travel starting to reopen, let's take a travel site as an example. By using faceted search, the site can present vacation options by destination region, trip type (i.e. hotel, self-catering, beach, ski, city break), price band, season, and more, enabling users to quickly navigate to the category that is most relevant to them. Facets also enable fast results counting. Extending our travel site example, business analysts can use facets to quickly compare sales statistics by counting the number of trips sold by region and season. Prior to the new facet operator, the only way Atlas Search could facet and count data was to retrieve the entire result set back to MongoDB’s internal $facet aggregation pipeline stage . While that was OK for smaller data sets, it became slow when the result set exceeded tens of thousands of documents. This all changes as now operations are pushed down to Atlas Search’s embedded and optimized Lucene library in a single $search pipeline stage. From our internal testing of a collection with one million documents, the new Atlas Search faceting improves performance by 100x. How to use faceting in Atlas Search Our new Atlas Search facets tutorial will help you get started. It describes how to: Create an index with a facet definition on string, date, and numeric fields in the sample_mflix.movies collection. Then run an Atlas Search query against those fields for results grouped by values for the string field and by ranges for the date and numeric fields, including the count for each of those groups. To use Atlas Search facets, you must be running your Atlas cluster on MongoDB 4.4.11 and above or MongoDB 5.0.4 and above. These clusters must be running on the M10 tier or higher. Facets and counts currently work on non-sharded collections. Support for sharded collections is scheduled for next year. The power of Atlas Search in a unified data platform in the cloud MongoDB Atlas Search makes it easy to build fast, relevant full-text search on top of your data in the cloud. A couple of API calls or clicks in the Atlas UI, and you instantly expose your data to sophisticated search experiences that boost engagement and improve satisfaction with your applications. Your data is immediately more discoverable, usable, and valuable. By embedding the Apache Lucene library directly alongside your database, data is automatically synchronized with the search index; developers get to work with a single API; there is no separate system to run and pay for; and everything is fully-managed for you on any cloud you choose. Figure 1: Rather than bolting-on a separate search engine to your database, Atlas Search provides a fully integrated platform. Atlas Search provides the power you get with Lucene — including faceted navigation, autocomplete, fuzzy search, built-in analyzers, highlighting, custom scoring, and synonyms — combining it with the productivity you get fromMongoDB. As a result, developers can ship search applications and new features 30%+ faster. Next steps You can try out Atlas Search with the public preview of lightning-fast facets and counts today: If you are new to Atlas Search, simply spin up a cluster (M10 tier or above) and get started with our Atlas Search facets tutorial . If you are already using Atlas Search on M10 tiers and above then update your indexes to use the facet field mapping , and then start querying ! Your data remains searchable while it is being re-indexed. If you want to dig into the use cases you can serve with Atlas Search — along with users who are already taking advantage of it today — download our new Atlas Search whitepaper . Safe Harbor The development, release, and timing of any features or functionality described for our products remains at our sole discretion. This information is merely intended to outline our general product direction and it should not be relied on in making a purchasing decision nor is this a commitment, promise or legal obligation to deliver any material, code, or functionality.
Fine-Tune Relevance in MongoDB Atlas Search with Function Scoring and Synonyms
MongoDB Atlas Search is an embedded full-text search solution in MongoDB Atlas that gives developers a seamless and scalable experience for building fast, relevance-based application features. We announced its general availability last year at MongoDB.live 2020 and over the past year we’ve introduced many new features, including a visual index builder, search query tester, custom analyzers , and wildcard path queries . This year at MongoDB.live 2021 , we’re excited to highlight two new capabilities that help developers tune the relevance of search results. See how easy it is to get started with MongoDB Atlas Search in this demo video by Marcus Eagan, Senior Product Manager for Atlas Search. Building relevance into search results Understanding the behavior of your users is essential when thinking about search result relevance. People don’t always tell you what they want, and they sometimes use words or phrases that don’t match your content exactly. To cover these scenarios, you can use full-text search features like function scoring and synonyms. Influence search rankings with function scoring There are often multiple factors that influence how search results should be ranked. For example, let’s say you have a restaurant finder application. The explicit inputs are things like the user’s location and what they’re searching for, but what’s implied is that they likely want to see highly rated restaurants or ones with more reviews. What’s Cooking: a sample restaurant finder application using MongoDB Atlas Search Function scoring allows you to influence the order of results returned by manipulating the score of each result. In Atlas Search, that means you can use a numeric field in a document and apply a mathematical expression to it. For example, you might want to increase the score of restaurants that are sponsored or have higher star ratings. This can easily be accomplished within the same search query by simply adding the function option to the score parameter of your query. Learn more about how to use function scores in our developer tutorial . Show results for more search queries with synonyms Synonyms are often used to define terms that are semantically similar to each other to improve search results. For example, someone searching for “noodles” might want to find results for “spaghetti”, “chow mein”, or “pad thai”. Synonyms can also help with typos, especially on mobile and small keyboards. In Atlas Search, you can define collections of synonyms for a search index via the API. Synonyms can be explicit (one-way) or equivalent (two-way). Explicit synonyms are good for defining relationships between terms that are subsets of each other, like the noodle example above: “spaghetti”, “chow mein”, and “pad thai” are all explicit synonyms for “noodles”, but not each other (you don’t want results for “chow mein” in a search for “spaghetti”). Equivalent synonyms are often used for terms that have regional variations or are otherwise interchangeable both ways, like soda and pop, or Kleenex and tissues. What's next for Atlas Search Developers are increasingly turning to full-text search to make content more discoverable and relevant for application end users. With Atlas Search, we hope to not only make building full-text search easier, but also more powerful and expressive. Join our community to ask questions and find out what other developers are building with Atlas Search and let us know what you think we should build next in our feedback forums .