Many modern search and retrieval augmented generation (RAG) applications that handle large volumes of multimodal, unstructured data require the precision of text search and the semantic similarity of vector search. Using either keyword search or vector search alone may not deliver optimally relevant results, resulting in unsatisfactory user experiences. Hybrid search works to provide a solution to this challenge.
Key takeaways
- Hybrid search combines lexical search and semantic search to deliver results that are both precise and contextually relevant.
- Lexical search excels at exact matching and predictable ranking, while semantic search excels at understanding user intent and natural language.
- Relying on only keyword search or only vector search can lead to incomplete or misleading results, especially in enterprise and AI-driven applications.
- Hybrid search improves result quality by merging BM25-based keyword relevance with vector similarity using techniques like reciprocal rank fusion (RRF) or relative score fusion (RSF).
- Use cases such as enterprise search, e-commerce discovery, content retrieval, and retrieval augmented generation (RAG) benefit significantly from hybrid search.
Table of contents
- What is hybrid search?
- What is lexical search?
- What is semantic search?
- What are the benefits of hybrid search?
- Lexical search vs. semantic search vs. hybrid search
- How hybrid search works
- Enhanced search and generative AI applications with hybrid search
- Considerations when implementing hybrid search
- Get started with hybrid search in MongoDB Atlas
- FAQs
- Related resources
What is hybrid search?
Hybrid search combines lexical search and semantic search to deliver results that are both precise and contextually relevant. Rather than choosing between keyword matching and vector similarity, hybrid search executes both approaches and intelligently merges the results. This allows systems to honor explicit constraints while still understanding user intent.
Hybrid search is particularly valuable in enterprise and generative AI applications where accuracy, explainability, and relevance must coexist.
While text (or keyword) search is effective in finding exact matches for query terms, semantic search provides the added benefit of identifying results with similar meanings even if they don't contain the exact query term. This ensures that contextually similar matches are also included in the combined results of both methods of search.
For example, a hybrid query like "healthy recipes with feta cheese" can benefit from both: keyword search ensures that the results contain feta cheese while semantic search looks beyond the actual "healthy" keyword and finds recipes that generally fit in the same healthy category, such as low-carb, low-sugar, or vegetable-focused.
Conversely, if there are specific terms or phrases within the dataset that should be explicitly omitted from the search results, hybrid search offers a valuable solution. By combining text search with semantic search, hybrid search ensures that these excluded keywords are effectively filtered out of the final results, delivering the most optimal experience to end users.
What is lexical search?
Lexical search, sometimes referred to as keyword or full-text search, retrieves documents by matching exact terms or phrases in a query to the terms stored in an index.
Lexical search excels at precision. It's highly effective when users know exactly what they are looking for, such as product names, IDs, error codes, or regulated terminology where wording matters.
Most modern lexical search systems rely on probabilistic ranking algorithms such as BM25, which evaluate relevance based on term frequency, document length, and inverse document frequency. This makes lexical search fast, predictable, and well-suited for filtering, compliance-driven queries, and deterministic search experiences.
Limitations emerge when user intent is vague, implicit, or expressed in natural language. Lexical search cannot infer meaning beyond the literal terms provided.
Learn more about lexical search.
What is semantic search?
Semantic search retrieves results based on meaning rather than exact wording. Instead of matching keywords, semantic search represents queries and documents as vector embeddings that capture contextual similarity.
When a user submits a query, it is transformed into a vector and compared against vectors stored in the database using similarity metrics such as cosine similarity or dot product. Results are ranked by conceptual closeness rather than term overlap.
Semantic search excels at handling natural language, ambiguous phrasing, synonyms, and exploratory queries. It enables use cases such as question answering, recommendation engines, and RAG.
Its limitation is precision. Semantic similarity alone can surface results that are contextually related but operationally incorrect or undesirable when strict constraints matter.
Learn more about semantic search.
What are the benefits of hybrid search?
Hybrid search applications can benefit organizations in multiple ways.
Enhanced user experiences
Hybrid search enhances user experience by understanding query intent, delivering context-focused search results, and reducing search effort, allowing users to quickly find the information they need.
Better insights
Analyzing search queries provides insights into user needs and preferences, helping organizations create higher value by identifying knowledge gaps and informing content creation and optimization strategies.
Increased efficiency
As hybrid search helps improve insight discovery, it also supports better decision-making and productivity by surfacing the right information at the right time.
Lexical search vs. semantic search vs. hybrid search
How hybrid search works
Hybrid search ranks and delivers the "best" results in terms of relevance from a keyword search result set and a vector search result set. Some popular and effective hybrid search techniques are discussed below.
Combining BM25 and vector search results
A popular approach that many hybrid search systems rely on is combining BM25 (Best Match 25) and vector similarity (measured using distance metrics like dot product, Euclidean, cosine, Hamming, etc.) scores to deliver more accurate and relevant search results.
BM25 is the default scoring algorithm in Apache Lucene and MongoDB Atlas Search. It focuses on keyword relevance and ranks documents based on the frequency in which the queried keywords appear, considering factors like document length and overall term frequency.
On the other hand, with vector search, documents are ranked based on their semantic relevance to the query rather than keyword matching. The user query is first converted into vector embeddings—numerical representations of the query text, which is then compared to the vector embeddings generated from all data within the database in order to find the closest matches. In MongoDB Atlas Vector Search, semantic similarity can be determined using Euclidean, cosine, or dot product metrics.
Finally, the BM25 and vector search scores are combined to create a unified ranking, delivering the highest-ranked results to end users.
There are multiple ways to combine the BM25 and vector search scores, among them are reciprocal rank fusion (RRF) and relative score fusion (RSF)—both of which are recommended techniques for hybrid search in MongoDB Atlas.
RRF works by calculating the reciprocal rank of each document across different search methods and then combining these ranks into a unified score for each document. It's particularly useful when there are various contextual meanings and data fields that need to be taken into consideration because it allows for a more balanced and well-rounded overall ranking.
RSF normalizes the scores from each search method, scaling them to a common range (typically 0 to 1), before combining them. This normalization ensures that the relative importance of each search method is preserved, even if their original score distributions differ. The normalized values can provide a more nuanced and accurate ranking of results compared to methods like RRF, which primarily focus on the order of results.
Combining dense and sparse vector search results
Another approach often leveraged by vector-only hybrid search solutions is using sparse and dense vectors to combine keyword and semantic relevance to improve search results.
Sparse vectors are primarily used for keyword-based search. They represent documents as a set of words or tokens, where each dimension corresponds to a specific word in the vocabulary. Sparse vectors excel at finding exact matches for query terms, making them suitable for tasks like filtering and precise information retrieval.
Dense vectors, generated by machine learning models trained on large amounts of data, capture the semantic meaning and context of documents. Dense vector search can handle complex queries, including those with natural language, ambiguous, or implicit terms.
Hybrid search then combines the scores from both sparse and dense vectors to deliver the final search results to the user, prioritizing the highest ones.
Enhanced search and generative AI applications with hybrid search
As explained above, hybrid search shines in scenarios where there's a need for both precision (where keyword search excels) and semantic search capabilities (where vector search excels), and where user queries can vary from simple to complex, including both keyword and natural language queries.
Organizations can implement hybrid search to deliver better results to end users via practical applications. Below are some common use cases where hybrid search can help improve search quality, and ultimately the user experience in a range of intelligent search and generative AI use cases.
Recommendation engines
Understanding users' interests and intent is a strong competitive advantage—like how Netflix provides a personalized selection of shows and movies based on your watch history, or how Amazon recommends products based on your purchase history. This is particularly powerful in e-commerce, media and entertainment, financial services, and product/service-oriented industries where the customer experience tightly influences the bottom line.
Retrieval augmented generation applications
With the rise of large language models (LLMs), search is increasingly becoming essential in generative AI and agentic AI application development. It augments LLMs by providing domain-specific context outside of what the LLMs “know,” ensuring relevance and accuracy of the gen AI output. In this case, hybrid search combining semantic search and text search can help the RAG model generate responses that are more contextually relevant, helping reduce hallucinations and improve the accuracy of the response.
E-commerce search engines
By combining keyword search and semantic search, hybrid search can better understand user intent and deliver tailored results. For instance, a customer searching for "women summer dress from XYZ brand" can receive results for various style and color combinations of dresses from that specific brand, even if they didn't explicitly mention those preferences.
Content search
As organizations' data grows, so does the need for more sophisticated, enhanced search, especially when there's a mix of structured (dates, locations, product IDs) and unstructured data (images, PDFs, audios, videos). Hybrid search makes it easier for users to locate relevant resources and obtain insights beyond a basic list of search results, particularly for queries that require advanced capabilities like semantic retrieval or natural language answers.
Learn how hybrid search is part of cognitive search.
Considerations when implementing hybrid search
There are several key factors to consider when planning and deploying a hybrid search system.
First, data quality is essential. High-quality data is the cornerstone of effective hybrid search. This involves ensuring data accuracy and consistency. Data cleaning and preprocessing are essential steps to remove errors, inconsistencies, and noise. Additionally, data enrichment can enhance search results by adding context and metadata to unstructured data like text, images, videos, and audios.
Second, architecture and techniques are also key. Since hybrid search relies on both text data and vector embeddings, it's important to select a vector database that is versatile, secure, and scalable with business needs. Choosing the right vector database means thoroughly evaluating key factors including the database's ability to provide performant indexing and querying capabilities, especially for real-time applications. The database should also seamlessly integrate with the organization's existing technology stack and be capable of handling large, growing datasets in various formats, all while maintaining a high level of security and database management support.
Further, it's worth understanding the techniques that will be applied to the hybrid search system, so that their strengths and weaknesses can be fully considered. A ranking technique that works well for certain datasets and use cases may need adjustments in order to work well for others.
And finally, as with any technology implementation, it's recommended that purposes and expected outcomes are defined prior to deployment, so that they can be measured and improved. For instance, how search quality is measured, how user feedback will be incorporated to improve results, whether the system delivers the metrics that it should, and more.
A well-implemented hybrid search system can deliver significant benefits, including improved user experiences thanks to more relevant results and better insights. And as the full potential of data is unlocked, this can also help enhance the overall organizational efficiency and decision making.
Get started with hybrid search in MongoDB Atlas
As organizations continue to innovate in the rapidly evolving technology ecosystem, building robust gen AI and search applications supporting customer, employee, and stakeholder experiences can deliver powerful competitive advantages.
With MongoDB, you can efficiently perform hybrid search, full-text search, and vector search, all natively within the database. Hybrid search combines the power of MongoDB Atlas Search and MongoDB Atlas Vector Search to deliver high-quality search results to support a variety of search and gen AI applications. There's no need to sync data for indexing and retrieval between your operational and vector databases—saving time, reducing complexity, and preventing errors. Your operational and vector data stay in one place.
Start building today—simplify your developer experience and create more value using the fully-managed, secure MongoDB developer data platform, integrated with a vast AI partner ecosystem, including all major cloud providers, LLM model providers, and system integrators.