For AI agents: a documentation index is available at https://www.mongodb.com/docs/llms.txt — markdown versions of all pages are available by appending .md to any URL path.
Docs Menu
Docs Home
/ /
Improve Accuracy
Docs Home
/
Development
/
Search
/
Improve Accuracy
Docs Home
/
Development
/
Search
/
Improve Accuracy

How to Perform Hybrid Search

You can combine MongoDB Vector Search and MongoDB Search queries into a hybrid search for unified results.

This tutorial demonstrates how to run a hybrid search on the sample_mflix.embedded_movies collection, which contains details about movies. Specifically, this tutorial takes you through the following steps:

  1. Create a MongoDB Vector Search index on the plot_embedding_voyage_4_large field. This field contains vector embeddings (of 2048 dimensions created using Voyage AI's voyage-4-large embedding model) that represent the summary of a movie's plot.

  2. Create a MongoDB Search index on the sample_mflix.embedded_movies collection.

  3. Run a query to combine the results from a $vectorSearch query and a $search query by using the following techniques:

    • Semantic boosting

    • Reciprocal rank fusion

    • Score Fusion

Why Hybrid Search?

A hybrid search is an aggregation of different search methods, such as a full-text and semantic search, for the same query criteria. While full-text is effective in finding exact matches for query terms, semantic search provides the added benefit of identifying semantically similar documents even if the documents don't contain the exact query term. This ensures that synonymous and contextually similar matches are also included in the combined results of both methods of search.

Conversely, if you have tokens for proper nouns or specific keywords in your dataset that you don't expect to be considered in the training of an embedding model in the same context that they are used in your dataset, your vector search might benefit from being combined with a full-text search.

You can also set weights for each method of search per query. Based on whether full-text or semantic search results are most relevant and appropriate for a query, you can increase the weight for that search method per query.

You can apply the following techniques to achieve a hybrid search:

Technique

Semantic Boosting

Reciprocal Rank Fusion

Relative Score Fusion

Purpose

Improves the accuracy of the full-text search results by boosting documents that match vector search. Useful when retrieving very large result sets.

Combines results from different search methods into a single ranked result set by calculating the reciprocal rank of documents in the results.

Combines results from different search methods into a single ranked result set by calculating the score of documents in the results.

Ranking Method

  • Run $vectorSearch to retrieves semantically similar results.

  • Run $search to retrieve full-text results while boosting the documents that were returned by $vectorSearch.

  • Retrieves $vectorSearch and $search results.

  • Merges both sets of results using the rank of each document in the respective result sets.

  • Retrieves $vectorSearch and $search results.

  • Merges both sets of results using the score of each document in the respective result sets.

Scoring Method

Boosts the full-text results score for documents that match the results from the vector search, by the weighted vector search score.

Calculates the score by using the following formula:

1.0/{document position in the results + constant value}

Then, for each method of search, applies different weights (w) to give more importance to that method of search. For each document, the weighted reciprocal rank is calculated by multiplying the weight by the reciprocal rank of the document.

weighted_reciprocal_rank = w x reciprocal_rank

Calculates the score by using the following formula:

score = w1 x score1 + w2 x score2

Search Features

  • Supports faceting to group data into categories.

  • Supports paginating results using searchAfter and searchBefore.

Supports further processing of results in subsequent aggregation stages such as $group.

Supports further processing of results in subsequent aggregation stages such as $group.

Output

Single result set that contains documents from $search

Single result set that contains documents from both search methods.

Single result set that contains documents from both search methods.

Prerequisites

Before you begin, you must have the following:

  • An Atlas cluster with MongoDB version v8.0 or higher.

    Note

    Ensure that your Atlas cluster has enough memory to store both MongoDB Search and MongoDB Vector Search indexes and run performant queries.

  • The sample_mflix data loaded into your Atlas cluster.

  • mongosh to run queries on your Atlas cluster:

  • Project Data Access Admin access to the project to create MongoDB Vector Search and MongoDB Search indexes.

Create the MongoDB Vector Search and MongoDB Search Indexes

This section demonstrates how to create the following indexes on the fields in the sample_mflix.embedded_movies collection:

  • A MongoDB Vector Search index on the plot_embedding_voyage_4_large field for running vector queries against that field.

  • A MongoDB Search index that dynamically indexes all dynamically indexable fields for running full-text search against those fields.

1

Connect to the cluster using mongosh.

To learn more, see Connect to a Cluster via mongosh.

2

Switch to the database that contains the collection for which you want to create the index.

use sample_mflix
switched to db sample_mflix
3

Define the MongoDB Vector Search index.

Run the following command. This index definition indexes the plot_embedding_voyage_4_large field as the MongoDB Vector Search field when querying the collection.

1db.embedded_movies.createSearchIndex(
2  "hybrid-vector-search", 
3  "vectorSearch", 
4  {
5    "fields": [
6      {
7        "type": "vector",
8        "path": "plot_embedding_voyage_4_large",
9        "numDimensions": 2048,
10        "similarity": "dotProduct"
11      }
12    ]
13  }
14);
4

Define the MongoDB Search index.

The following index definition automatically indexes all the dynamically indexable fields in the collection.

1db.embedded_movies.createSearchIndex(
2  "hybrid-full-text-search", 
3  "search", 
4  {
5    "mappings": {
6      "dynamic": true
7    }
8  }
9);

Run the Hybrid Search Queries

This section demonstrates how to query the data in the sample_mflix.embedded_movies collection for charming animal in the plot_embedding_voyage_4_large and fullplot fields by using the $vectorSearch and $search pipeline stages, respectively. The query enhances relevancy of $search queries by boosting the full-text scores using $vectorSearch scores for the same documents.

Semantic Boosting Query

1

Create the file for the embeddings.

  1. Create a file named query-embeddings.js.

    touch query-embeddings.js

  2. Copy and paste the following embeddings into the file.

    CHARMING_ANIMAL_EMBEDDING=[-0.02300029993057251, -0.040718235075473785, 0.026521876454353333, -0.007813499309122562, 0.01958877220749855, -0.032134391367435455, 0.015186801552772522, 0.023990744724869728, -0.0029025499243289232, -0.0019671309273689985, 0.01254561822861433, -0.016287293285131454, -0.029053010046482086, 0.01408630795776844, -0.00891399197280407, 0.01958877220749855, -0.0508427694439888, -0.015737047418951988, 0.012215470895171165, -0.0036316262558102608, 0.0033152345567941666, 0.0033840155228972435, -0.00748335151001811, 0.002737476024776697, 0.03257458657026291, -0.01892847754061222, -0.012050396762788296, 0.02432089112699032, 0.028392715379595757, 0.033454980701208115, 0.005144803784787655, 0.0072632525116205215, 0.0212395116686821, -0.01683754101395607, -0.002751232124865055, 0.030813800171017647, -0.025861581787467003, 0.00748335151001811, -0.0032326977234333754, -0.030813800171017647, 0.006768031045794487, 0.024651039391756058, -0.03565596789121628, -0.02575153298676014, -0.005970173515379429, 0.02575153298676014, -0.08011587709188461, 0.0022284979932010174, -0.03257458657026291, 0.00522734085097909, -0.0028062567580491304, 0.011059952899813652, 0.0061627598479390144, 0.021349560469388962, -0.0061627598479390144, 0.02200985699892044, 0.006988129578530788, -0.006850568111985922, 0.010784829966723919, -0.0035490894224494696, -0.007538375910371542, 0.0023110350593924522, -0.031033897772431374, -0.04291922226548195, -0.03389517962932587, -0.004181872587651014, -0.02024906873703003, 0.003769187955185771, -0.01650739274919033, -0.0031226484570652246, -0.0026686950586736202, 0.03389517962932587, 0.020689265802502632, -0.025531433522701263, 0.02024906873703003, 0.03125399723649025, 0.006602956913411617, -0.01441645622253418, -0.04974227771162987, 0.005887636449187994, -0.013150889426469803, 0.007593400776386261, 0.00352157698944211, -0.014196357689797878, -0.009849410504102707, -0.019698821008205414, -0.01441645622253418, 0.0018708378775045276, -0.007703450042754412, 0.0020771801937371492, -0.03565596789121628, 0.027402272447943687, 0.007593400776386261, 0.005695050582289696, 0.03389517962932587, 0.014196357689797878, -0.002682451391592622, 0.030373603105545044, -0.01061975583434105, -0.017387786880135536, -0.00522734085097909, -0.013205913826823235, 0.007813499309122562, 0.02509123645722866, 0.00304011139087379, -0.04225892573595047, -0.011500149965286255, 0.005887636449187994, 0.0016163488617166877, -0.03477557376027107, 0.04423981159925461, -0.005144803784787655, -0.004401971120387316, -0.028062567114830017, 0.0087489178404212, 0.0029850867576897144, -0.0062177847139537334, -0.012215470895171165, -0.001430640695616603, 0.017277738079428673, -0.007813499309122562, -0.03191429376602173, 0.033454980701208115, 0.018818426877260208, 0.0035766016226261854, -0.030373603105545044, -0.056125134229660034, -0.01815813221037388, 0.0031501606572419405, 0.013756160624325275, -0.03785695508122444, -0.011775272898375988, 0.04908198118209839, 0.031474094837903976, -0.0004126848070882261, 0.027732418850064278, 0.01716768741607666, 0.005640025716274977, -0.013646110892295837, -0.010124534368515015, -0.005007242318242788, -0.02300029993057251, 0.027622370049357414, 0.015737047418951988, -0.03323488309979439, -0.00018312888278160244, -0.010509707033634186, -0.0024898650590330362, -0.014746603555977345, -0.004704606719315052, 0.03191429376602173, 0.02443094179034233, -0.0013756160624325275, -0.0046220701187849045, -0.049962375313043594, -0.01782798394560814, 0.0003920505696441978, -0.01892847754061222, 0.0012311763130128384, 0.041378531605005264, -0.041378531605005264, -0.02090936340391636, -0.010234583169221878, -0.008418770506978035, 0.03477557376027107, 0.009684337303042412, 0.05194326117634773, -0.010344632901251316, -0.047541290521621704, 0.010509707033634186, -0.03477557376027107, -0.023440496996045113, -0.007153203245252371, 0.002737476024776697, 0.020689265802502632, 0.020689265802502632, 0.03301478549838066, 0.012490593828260899, -0.024210842326283455, -0.0216797087341547, 0.005062267184257507, 0.0032877223566174507, -0.02949320711195469, -0.00232479115948081, -0.023990744724869728, 0.04534030333161354, 0.010124534368515015, 0.015516948886215687, 0.04490010812878609, -0.0007084422395564616, -0.04842168465256691, -0.025531433522701263, 0.0216797087341547, -0.010399657301604748, 0.049962375313043594, -0.02024906873703003, 0.03477557376027107, 0.008803942240774632, 0.02057921513915062, -0.01408630795776844, -0.06074720621109009, -0.010014484636485577, 0.034115277230739594, -0.017387786880135536, -0.02410079352557659, -0.01100492849946022, -0.02575153298676014, -0.026521876454353333, -0.03741675615310669, 0.011610199697315693, -0.045560404658317566, 0.0005811977898702025, 0.016067195683717728, 0.008473794907331467, -0.02024906873703003, -0.03961774334311485, 0.00748335151001811, 0.02784246951341629, 0.021349560469388962, 0.02333044819533825, 0.009574287571012974, 0.0006431004730984569, -0.035876065492630005, 0.002022155560553074, 0.019368674606084824, -0.004181872587651014, -0.011335075832903385, -0.02883291244506836, -0.000839125772472471, 0.006713006179779768, -0.00030263554072007537, -0.0025724018923938274, -0.013811185024678707, 0.001843325444497168, -0.013701136223971844, -0.017277738079428673, -0.006245296914130449, -0.006933104712516069, -0.014636554755270481, 0.0033977716229856014, 0.004044311121106148, -0.01584709621965885, 0.00858384370803833, -0.0042093852534890175, 0.0005124169983901083, -0.013315963558852673, -0.03301478549838066, -0.011059952899813652, -0.02057921513915062, 0.0010385900968685746, -0.002421084325760603, 0.00043503858614712954, 0.02443094179034233, 0.009794386103749275, -0.002751232124865055, 0.006382858380675316, 0.018488280475139618, 0.03565596789121628, -0.021789757534861565, 0.011830298230051994, -0.021569659933447838, -0.00034906258224509656, -0.032134391367435455, 0.015406900085508823, -0.02432089112699032, -0.004181872587651014, 0.010509707033634186, 0.020689265802502632, 0.04468001052737236, -0.018048083409667015, -0.00014701895997859538, -0.036536362022161484, 0.02993340604007244, 0.022890251129865646, -0.024210842326283455, 0.02366059646010399, 0.0005777587648481131, 0.011775272898375988, -0.0349956713616848, 0.05458444356918335, 0.004842168651521206, 0.013646110892295837, 0.022780202329158783, -0.006520419847220182, -0.008803942240774632, -0.0035766016226261854, 0.052383460104465485, -0.011940347030758858, 0.0027925006579607725, 0.011280051432549953, 0.029053010046482086, 0.015406900085508823, 0.009959460236132145, -0.0011899078963324428, 0.035876065492630005, -0.001671373494900763, 0.006575444713234901, -0.014306406490504742, 0.001011077780276537, -0.020689265802502632, -0.021129462867975235, -0.004704606719315052, 0.003342746989801526, -0.005832612048834562, -0.0025724018923938274, -0.003769187955185771, -0.005447439383715391, 0.027952518314123154, 0.011830298230051994, 0.03961774334311485, 0.0550246424973011, 0.0031364045571535826, 0.0012311763130128384, -0.002764988224953413, -0.05150306597352028, -0.02608167938888073, 0.015296850353479385, -0.0020634240936487913, -0.03675645962357521, 0.026962075382471085, 0.023770645260810852, -0.011940347030758858, -0.05898641422390938, -0.03785695508122444, -0.003026355290785432, 0.0021322048269212246, 0.02509123645722866, -0.01925862394273281, 0.03961774334311485, 0.02333044819533825, -0.00031123313237912953, 0.06206779554486275, 0.00010102180385729298, -0.01683754101395607, -0.02751232124865055, 0.03851724788546562, 0.0073182773776352406, 0.02641182765364647, -0.042699120938777924, -0.008969016373157501, -0.01287576649338007, 0.018268181011080742, 0.01991892047226429, -0.002916306024417281, 0.032794687896966934, -0.03873734921216965, 0.023550545796751976, -0.03191429376602173, -0.0072632525116205215, 0.0022972787264734507, 0.023440496996045113, -0.01441645622253418, 0.0035766016226261854, 0.016287293285131454, 0.021569659933447838, -0.0004505142569541931, -0.0028062567580491304, -0.004099335987120867, 0.015516948886215687, 0.02024906873703003, 0.004787143785506487, 0.03477557376027107, -0.009244140237569809, 0.026852024719119072, -0.002778744325041771, 0.003741675755009055, 0.015296850353479385, 0.029053010046482086, -0.020139018073678017, 0.006492907647043467, -0.02575153298676014, 0.03983784094452858, 0.0038242125883698463, 0.010344632901251316, -0.012490593828260899, -0.02333044819533825, -0.019808871671557426, -0.04490010812878609, 0.013205913826823235, -0.037636853754520416, 0.00891399197280407, 0.0011555175296962261, 0.022450054064393044, 0.04225892573595047, 0.0018158131279051304, 0.004869680851697922, 0.02949320711195469, 0.01496670302003622, -0.013150889426469803, -0.04776138812303543, -0.020359117537736893, -0.011665224097669125, -0.00464958231896162, -0.016287293285131454, -0.024651039391756058, -0.005887636449187994, -0.0020634240936487913, 0.0014581530122086406, -0.020028969272971153, -0.002861281391233206, 0.0072632525116205215, -0.022890251129865646, -0.027292221784591675, 0.017938032746315002, 0.03125399723649025, -0.012600642628967762, -0.02333044819533825, 0.03477557376027107, -0.027072124183177948, 0.022780202329158783, -0.010509707033634186, 0.02784246951341629, -0.02057921513915062, -0.010949904099106789, -0.011830298230051994, -0.023550545796751976, 0.005364902783185244, 0.02476108819246292, 0.0002820013032760471, 0.0036591386888176203, 0.04423981159925461, 0.022780202329158783, 0.047981489449739456, -0.02200985699892044, 0.009904435835778713, 0.0036316262558102608, -0.007813499309122562, 0.02608167938888073, 0.014306406490504742, 0.011114977300167084, -0.040057938545942307, 0.01782798394560814, -0.022450054064393044, 0.043799616396427155, 0.0212395116686821, -0.022119905799627304, -0.015626998618245125, 0.013811185024678707, 0.026852024719119072, 0.0029025499243289232, -0.01782798394560814, 0.005777587182819843, 0.03851724788546562, -0.047981489449739456, 0.011610199697315693, 0.037636853754520416, 0.01408630795776844, -0.026191730052232742, 0.025311335921287537, 0.04181872680783272, 0.027402272447943687, -0.004512020852416754, -0.044459909200668335, 0.004236897453665733, -0.022229954600334167, -0.005309877917170525, 0.005254853516817093, 0.002517377259209752, 0.016067195683717728, 0.02476108819246292, -0.001341225695796311, -0.001341225695796311, 0.023440496996045113, -0.014746603555977345, 0.00748335151001811, 0.03389517962932587, -0.014031283557415009, -0.04886188358068466, -0.007813499309122562, -0.03389517962932587, 0.034115277230739594, -0.00522734085097909, 0.03807705268263817, -0.013095865026116371, -0.00619027204811573, 0.025641482323408127, 0.023770645260810852, -0.02751232124865055, 0.013315963558852673, 0.030373603105545044, -0.01958877220749855, 0.007923548109829426, -0.016727490350604057, 0.01782798394560814, -0.008693893440067768, -0.01496670302003622, 0.020139018073678017, -0.007703450042754412, -0.010069509036839008, 0.006602956913411617, 0.05216335877776146, 0.022450054064393044, 0.017607884481549263, -0.014526505023241043, -0.020689265802502632, 0.003961774054914713, 0.0020496679935604334, 0.013150889426469803, -0.035215768963098526, -0.024651039391756058, -0.02432089112699032, -0.013811185024678707, 0.031474094837903976, -0.034555476158857346, 0.006933104712516069, 0.038957446813583374, 0.013811185024678707, -0.008803942240774632, -0.02993340604007244, 0.01254561822861433, 0.010894878767430782, -0.011665224097669125, -0.0005674416315741837, -0.02101941406726837, 0.02883291244506836, -0.025201285257935524, 0.0007978572975844145, 0.003934261854737997, -0.004484508186578751, -0.02883291244506836, -0.02883291244506836, 0.0014031283790245652, -0.008803942240774632, -0.01287576649338007, -0.0010661024134606123, 0.025971630588173866, -0.024871138855814934, 0.020799314603209496, 0.010124534368515015, -0.03609616681933403, -0.026191730052232742, -0.015737047418951988, -0.003205185290426016, 0.0174978356808424, 0.03565596789121628, 0.02641182765364647, -0.006547932513058186, -0.00720822811126709, -0.005777587182819843, 0.007098178844898939, 0.03257458657026291, -0.020799314603209496, 0.0073182773776352406, -0.0216797087341547, 0.010949904099106789, -0.0010661024134606123, 0.00522734085097909, 0.011500149965286255, 0.0254213847219944, 0.008473794907331467, 0.01238054409623146, 0.011390101164579391, 0.013811185024678707, 0.0033014784567058086, -0.009684337303042412, -0.00453953305259347, 0.0012999571627005935, 0.008253696374595165, -0.0063553461804986, 0.0035766016226261854, 0.015737047418951988, -0.024540990591049194, 0.016947589814662933, 0.016397343948483467, -0.005144803784787655, 0.013260938227176666, 0.024981187656521797, -0.0073182773776352406, -0.038297150284051895, -0.016067195683717728, -0.006410370580852032, -0.011225027032196522, 0.012820741161704063, -0.009189114905893803, -0.01925862394273281, -0.02993340604007244, 0.012105421163141727, -0.0254213847219944, -0.006575444713234901, -0.02817261591553688, -0.025861581787467003, -0.012655667960643768, 0.034115277230739594, -0.03059370070695877, -0.00748335151001811, -0.006382858380675316, -0.005832612048834562, -0.015186801552772522, -0.01408630795776844, 0.029053010046482086, -0.026962075382471085, 0.012105421163141727, -0.056125134229660034, -0.004759631585329771, 0.0023798157926648855, 0.0016094708116725087, -0.01441645622253418, 0.0783550888299942, 0.026191730052232742, -0.013646110892295837, -0.04357951506972313, -0.019808871671557426, -0.0011486393632367253, -0.0038792372215539217, -0.017607884481549263, 0.012215470895171165, 0.03675645962357521, 0.044459909200668335, 0.017277738079428673, -0.029053010046482086, 0.0026962074916809797, 0.01045468170195818, -0.023110348731279373, 0.008363745175302029, -0.02090936340391636, 0.019698821008205414, -0.018378229811787605, -0.016727490350604057, 0.030813800171017647, 0.03301478549838066, -0.055464837700128555, 0.031474094837903976, 0.022229954600334167, -0.019478723406791687, -0.015186801552772522, 0.02333044819533825, 0.006850568111985922, 0.0024623526260256767, 0.03301478549838066, -0.012490593828260899, -0.017938032746315002, 0.0010454681469127536, -0.009134090505540371, -0.054144248366355896, 0.02333044819533825, 0.03477557376027107, -0.026631927117705345, -0.02674197591841221, -0.005062267184257507, -0.010179558768868446, -0.0005708806565962732, 0.01408630795776844, 0.011114977300167084, 0.01859832927584648, -0.0018708378775045276, 0.0016369831282645464, -0.01083985436707735, -0.019478723406791687, 0.022780202329158783, -0.004126848187297583, 0.032134391367435455, 0.02024906873703003, 0.007813499309122562, 0.00018656792235560715, 0.01859832927584648, -0.004512020852416754, 0.004126848187297583, -0.015626998618245125, -0.018268181011080742, -0.007098178844898939, -0.015626998618245125, -0.01815813221037388, 0.013921234756708145, 0.013811185024678707, 0.016617441549897194, 0.03543587028980255, 0.02024906873703003, 0.0010661024134606123, -0.015076751820743084, 0.003769187955185771, -0.011940347030758858, 0.0061627598479390144, -0.04401971399784088, -0.005640025716274977, -0.0009560531470924616, -0.023440496996045113, 0.03873734921216965, 0.005062267184257507, 0.01716768741607666, 0.001760788494721055, 0.002256010426208377, 0.013095865026116371, 0.011500149965286255, 0.019038526341319084, -0.006382858380675316, -0.0433594174683094, -0.0072632525116205215, -0.022119905799627304, 0.036536362022161484, 0.019698821008205414, 0.0018845939775928855, -0.029053010046482086, -0.004787143785506487, -0.02883291244506836, 0.005942661315202713, 0.045120205730199814, 0.0036316262558102608, 0.011665224097669125, 0.008253696374595165, 0.02509123645722866, 0.03323488309979439, 0.02718217298388481, -0.013040839694440365, 0.01782798394560814, 0.029053010046482086, -0.011280051432549953, 0.01216044556349516, -0.014196357689797878, 0.005640025716274977, 0.03059370070695877, -0.010124534368515015, -0.000997321680188179, -0.0004883437068201602, 0.004787143785506487, 0.009354189038276672, -0.007015641778707504, 0.009849410504102707, -0.02410079352557659, -0.0062177847139537334, -0.0031364045571535826, 0.004952217917889357, 0.020469166338443756, -0.029273109510540962, -0.01782798394560814, 0.004787143785506487, -0.024540990591049194, -0.012435569427907467, 0.017057638615369797, -0.0037967003881931305, 0.02267015166580677, -0.011335075832903385, 0.002448596525937319, 0.045560404658317566, 0.03125399723649025, 0.022229954600334167, 0.05898641422390938, 0.028392715379595757, 0.004044311121106148, -0.04181872680783272, 0.000254488957580179, 0.01254561822861433, -0.015737047418951988, -0.01496670302003622, 0.01925862394273281, 0.049302078783512115, -0.012325519695878029, -0.0017263981280848384, -0.0008425648557022214, 0.019038526341319084, -0.0073182773776352406, 0.018268181011080742, 0.030153503641486168, 0.055464837700128555, 0.04490010812878609, 0.019038526341319084, 0.04225892573595047, -0.022119905799627304, -0.015296850353479385, -0.0009285408305004239, -0.03741675615310669, -0.03961774334311485, 0.005254853516817093, 0.033454980701208115, 0.006272809114307165, 0.010894878767430782, -0.010069509036839008, 0.0026411828584969044, -0.010014484636485577, -0.020139018073678017, -0.01617724448442459, 0.015626998618245125, -0.010014484636485577, 0.03961774334311485, 0.0216797087341547, -0.004594557452946901, -0.00464958231896162, -0.006547932513058186, 0.029273109510540962, -0.009409213438630104, 0.015406900085508823, -0.011445125564932823, -0.013646110892295837, 0.008088622242212296, -0.023880694061517715, 0.013921234756708145, 0.005887636449187994, 0.05216335877776146, 0.011445125564932823, 0.013260938227176666, -0.046220697462558746, -0.019808871671557426, -0.008198671974241734, 0.016947589814662933, -0.020799314603209496, 0.02476108819246292, 0.0017883008113130927, -0.034115277230739594, -0.017607884481549263, -0.028612812981009483, 0.03983784094452858, -0.022780202329158783, -0.002256010426208377, 0.032794687896966934, 0.04093833267688751, 0.022229954600334167, 0.0002157997660106048, 0.013150889426469803, 0.0024761089589446783, -0.005832612048834562, 0.02993340604007244, 0.006933104712516069, -0.038957446813583374, -0.013756160624325275, -0.01584709621965885, -0.0025724018923938274, 0.011610199697315693, 0.00020634240354411304, 0.03191429376602173, 0.0019396186107769608, -0.016947589814662933, -0.049962375313043594, -0.04225892573595047, 0.0022284979932010174, -0.034555476158857346, -0.04974227771162987, -0.00352157698944211, 0.026852024719119072, -0.027732418850064278, -0.04115843027830124, -4.126848216401413e-05, 0.01683754101395607, 0.006272809114307165, 0.011059952899813652, -0.015516948886215687, 0.004181872587651014, -0.015626998618245125, 0.0021046926267445087, 0.03565596789121628, -0.038297150284051895, -0.01925862394273281, -0.054144248366355896, -0.02949320711195469, -0.003342746989801526, 0.026301778852939606, -0.013701136223971844, 0.005254853516817093, 0.021349560469388962, -0.013040839694440365, -0.023990744724869728, -0.018708378076553345, 0.012930790893733501, -0.006492907647043467, -0.025861581787467003, -0.02333044819533825, -0.02949320711195469, -0.014856653288006783, -0.017387786880135536, -0.020799314603209496, 0.013701136223971844, -0.008033597841858864, -0.034555476158857346, -0.0007841011392883956, -0.006740518845617771, -0.024981187656521797, -0.020799314603209496, -0.016067195683717728, 0.013426012359559536, -0.02333044819533825, -0.003906749654561281, -0.00608022278174758, 0.03873734921216965, -0.033454980701208115, -0.03873734921216965, 0.02817261591553688, 0.01716768741607666, -0.02509123645722866, -0.010399657301604748, 0.013811185024678707, 0.029053010046482086, -0.044459909200668335, 6.878080057504121e-06, -0.007868523709475994, 0.01254561822861433, 0.008143646642565727, 0.036536362022161484, 0.015957146883010864, 0.021789757534861565, -0.01083985436707735, -0.0424790233373642, -0.0014581530122086406, -0.008033597841858864, 0.0036591386888176203, 0.026852024719119072, 0.006630469113588333, 0.010674780234694481, -0.01650739274919033, 0.04401971399784088, -0.018708378076553345, 0.032134391367435455, -0.00453953305259347, 0.0005640026065520942, 0.008253696374595165, -0.030373603105545044, 0.03257458657026291, 0.007868523709475994, -0.0069055925123393536, 0.03059370070695877, 0.04842168465256691, 0.014636554755270481, 0.011390101164579391, 0.04291922226548195, 0.001169273629784584, 0.02200985699892044, -0.007593400776386261, 0.02509123645722866, -0.0508427694439888, 0.008363745175302029, 0.017387786880135536, 0.04490010812878609, -0.015626998618245125, 0.01045468170195818, 0.021789757534861565, 0.011720248498022556, -0.033454980701208115, 0.00453953305259347, -0.029053010046482086, 0.04468001052737236, -0.031033897772431374, 0.00018312888278160244, -0.005942661315202713, -0.010344632901251316, 0.04225892573595047, -0.009904435835778713, -0.05942661315202713, -0.009189114905893803, 0.025861581787467003, -0.01216044556349516, -0.004346946720033884, 0.013315963558852673, -0.008803942240774632, 0.0046220701187849045, -0.019368674606084824, 0.026191730052232742, 0.012050396762788296, 0.01815813221037388, -0.025641482323408127, -0.0012449325295165181, 0.0006327833980321884, -0.02751232124865055, 0.03389517962932587, 0.02443094179034233, 0.006300321314483881, -0.013370987959206104, 0.006575444713234901, 0.026631927117705345, 0.017387786880135536, -0.028062567114830017, -0.005887636449187994, 0.00021064121392555535, 0.019478723406791687, -0.00018656792235560715, 0.00891399197280407, 0.02443094179034233, 0.008638869039714336, -0.029273109510540962, -0.015406900085508823, 0.007373301777988672, -0.0011899078963324428, -0.0034665523562580347, -0.00304011139087379, 0.012765716761350632, 0.032134391367435455, 0.023110348731279373, -0.027952518314123154, 0.0007325155311264098, -0.05150306597352028, -0.018488280475139618, 0.016397343948483467, 0.0024073279928416014, -0.032134391367435455, -0.002269766526296735, 0.053704049438238144, -0.007098178844898939, -0.022560102865099907, -0.028062567114830017, -0.009794386103749275, -0.04159862920641899, -0.020359117537736893, 0.016287293285131454, -0.005915149115025997, -0.011995372362434864, -0.012325519695878029, -0.019148575142025948, -0.03675645962357521, 0.015406900085508823, 0.008803942240774632, -0.019478723406791687, -0.02509123645722866, 0.025641482323408127, -0.014306406490504742, -0.013811185024678707, -0.06558937579393387, -0.023110348731279373, -0.002022155560553074, 0.025531433522701263, -0.004759631585329771, -0.04291922226548195, 0.006107735447585583, 0.011335075832903385, 0.046220697462558746, -0.03059370070695877, 0.006768031045794487, 0.054144248366355896, -0.018378229811787605, 0.001843325444497168, 0.005447439383715391, -0.03961774334311485, 0.00858384370803833, 0.01441645622253418, -0.01408630795776844, 0.060307007282972336, 0.021569659933447838, 0.047981489449739456, 0.010124534368515015, 0.0174978356808424, -0.028392715379595757, -0.03125399723649025, 0.019148575142025948, 0.01892847754061222, 0.0072632525116205215, 0.02057921513915062, -0.008198671974241734, 0.0011555175296962261, -0.00946423877030611, 0.009024041704833508, -0.02883291244506836, 0.018818426877260208, -0.0029713306576013565, -0.014636554755270481, 0.011059952899813652, -0.02718217298388481, 0.013536062091588974, 0.003507820889353752, 0.029053010046482086, 0.0031776730902493, -0.016727490350604057, -0.012270495295524597, -0.04886188358068466, -0.0004212824278511107, 0.04732119292020798, -0.001258688629604876, 0.0003628187405411154, -0.024871138855814934, -0.005585001315921545, -0.0041543603874742985, 0.03785695508122444, -0.008528819307684898, 0.010399657301604748, 0.015076751820743084, -0.022780202329158783, -0.005997686181217432, 0.032794687896966934, 0.029713306576013565, 0.0174978356808424, -0.015626998618245125, -0.006327833980321884, 0.003906749654561281, 0.019368674606084824, -0.014306406490504742, -0.04115843027830124, 0.0216797087341547, -0.03323488309979439, -0.010344632901251316, -0.0045670452527701855, 0.02718217298388481, 0.005529976449906826, 0.004484508186578751, 0.021459611132740974, 0.007373301777988672, -0.002737476024776697, 0.014526505023241043, 0.0015750803286209702, -0.0508427694439888, -0.010399657301604748, 0.0016438611783087254, -0.014526505023241043, -0.02949320711195469, -0.016287293285131454, 0.03807705268263817, -0.005997686181217432, -0.0027237196918576956, 0.03059370070695877, 0.05062267184257507, 0.03565596789121628, -0.02200985699892044, -0.026631927117705345, -0.01859832927584648, -0.006657981779426336, -0.025641482323408127, 0.014306406490504742, -0.0007290765061043203, 0.002448596525937319, -0.009794386103749275, 0.02509123645722866, -0.00929916463792324, 0.0026686950586736202, 0.04710109159350395, -0.028392715379595757, -0.007373301777988672, 0.03301478549838066, -0.009629311971366405, -0.026521876454353333, -0.026631927117705345, -0.03609616681933403, 0.03367508202791214, 0.0174978356808424, -0.010069509036839008, -0.02883291244506836, 0.02057921513915062, -0.002847525291144848, 0.007648425176739693, 0.0057500749826431274, 0.045560404658317566, -0.008198671974241734, 0.029053010046482086, 0.010894878767430782, 0.00891399197280407, 0.008693893440067768, 0.013150889426469803, -0.017277738079428673, 0.0016369831282645464, -0.003961774054914713, -0.03873734921216965, -0.004677094519138336, -0.017938032746315002, 0.012105421163141727, -0.009904435835778713, -0.011830298230051994, 0.005695050582289696, -0.029273109510540962, 0.00720822811126709, 0.02476108819246292, -0.030813800171017647, -0.03301478549838066, -0.0033014784567058086, 0.012270495295524597, 0.042699120938777924, -0.028392715379595757, -0.008858967572450638, -0.03961774334311485, -0.008198671974241734, 0.017607884481549263, -0.008033597841858864, 0.03719665855169296, 0.0017057638615369797, -0.01061975583434105, -0.037636853754520416, 0.023880694061517715, 0.011885322630405426, 0.018488280475139618, 0.014526505023241043, 0.01782798394560814, -0.02333044819533825, 0.007153203245252371, 0.008198671974241734, -0.015406900085508823, 0.008693893440067768, 0.020359117537736893, -0.01441645622253418, -0.05458444356918335, 0.014636554755270481, -0.01408630795776844, 0.005915149115025997, 0.000832247722428292, -0.0011280052131041884, -0.0017745447112247348, -0.005860124249011278, 0.0005536854732781649, -0.0046220701187849045, 0.024651039391756058, 0.013756160624325275, -0.006025198381394148, -0.0433594174683094, 0.02674197591841221, -0.009189114905893803, -0.00464958231896162, -0.016287293285131454, -0.0028200128581374884, -0.04159862920641899, 0.0022422540932893753, -0.06426878273487091, 0.00720822811126709, 0.015076751820743084, -0.015296850353479385, 0.009904435835778713, -0.0349956713616848, 0.008308720774948597, -0.029713306576013565, -0.00907906610518694, -0.011830298230051994, -0.007373301777988672, 0.008528819307684898, 0.02575153298676014, 0.04203882813453674, 0.0087489178404212, -0.024871138855814934, 0.02300029993057251, -0.024540990591049194, -0.02234000526368618, 0.019698821008205414, -0.009354189038276672, -0.02641182765364647, -0.00352157698944211, -0.02641182765364647, -0.0011830298462882638, 0.04049813747406006, -0.008803942240774632, -0.011885322630405426, -0.012050396762788296, -0.025971630588173866, 0.010179558768868446, -0.015296850353479385, -0.006878080312162638, 0.015406900085508823, -0.01892847754061222, -0.030153503641486168, -0.01238054409623146, -0.0433594174683094, 0.021789757534861565, 0.0027237196918576956, -0.024871138855814934, -0.00720822811126709, -0.003604114055633545, 0.005199828650802374, -0.006740518845617771, -0.0006499785813502967, -0.03741675615310669, 0.001506299595348537, -0.007923548109829426, 0.015626998618245125, -0.005832612048834562, 0.008143646642565727, -0.05898641422390938, -0.006272809114307165, 0.0038792372215539217, -0.020689265802502632, 0.020028969272971153, -0.017057638615369797, 0.030813800171017647, -0.011830298230051994, 0.017277738079428673, 0.013095865026116371, 0.012930790893733501, -0.012105421163141727, 0.010344632901251316, -0.0005743196816183627, -0.007153203245252371, -0.023110348731279373, 0.008088622242212296, -0.0008528819307684898, -0.011555174365639687, 0.010729805566370487, -0.01650739274919033, 0.023550545796751976, -0.0049247052520513535, -0.006878080312162638, 0.00464958231896162, -0.08055607229471207, 0.012105421163141727, -0.024981187656521797, -0.05590503662824631, 0.0018570816610008478, 0.0021046926267445087, -0.021349560469388962, 0.013976259157061577, -0.05018247291445732, -0.012655667960643768, 0.018048083409667015, -0.014636554755270481, 0.014306406490504742, 0.008858967572450638, 0.025531433522701263, -0.0043744589202106, -0.004044311121106148, -3.202606239938177e-05, 0.016067195683717728, -0.014746603555977345, 0.03741675615310669, -0.023880694061517715, 0.010179558768868446, -0.02366059646010399, 0.04401971399784088, -0.027622370049357414, -0.0049247052520513535, -0.008528819307684898, 0.017607884481549263, -0.018488280475139618, -0.022450054064393044, -0.01584709621965885, -0.017607884481549263, 0.01859832927584648, 0.023880694061517715, 0.019368674606084824, -0.008803942240774632, -0.0019533748272806406, 0.03631626442074776, -0.009189114905893803, 0.0216797087341547, -0.0014719091122969985, 0.019698821008205414, -0.011114977300167084, -0.017057638615369797, 0.00453953305259347, 0.04908198118209839, 0.002998843090608716, -0.0013893721625208855, 0.009409213438630104, -0.047541290521621704, -0.03433537483215332, -0.002159717259928584, -0.011280051432549953, 0.010894878767430782, 0.02949320711195469, 0.0072632525116205215, -5.502464046003297e-05, -0.015626998618245125, 0.01782798394560814, 0.012490593828260899, 0.005474952049553394, 0.028612812981009483, 0.012050396762788296, 0.020028969272971153, 0.021789757534861565, 0.011885322630405426, 0.005997686181217432, 0.020359117537736893, 0.01100492849946022, -0.008198671974241734, 0.007868523709475994, -0.030153503641486168, 0.013811185024678707, 0.009024041704833508, -0.006740518845617771, 0.003163916990160942, 0.024651039391756058, -0.032134391367435455, -0.0016851297114044428, 0.004869680851697922, -0.010894878767430782, 0.004512020852416754, 0.008033597841858864, 0.003961774054914713, 0.027952518314123154, 0.018378229811787605, -0.011775272898375988, 0.003026355290785432, -0.015737047418951988, 0.00946423877030611, 0.019478723406791687, 0.0032877223566174507, 0.009849410504102707, -0.004787143785506487, -0.02509123645722866, -0.019698821008205414, 0.0006602957146242261, -0.0316941924393177, 0.017938032746315002, 0.006713006179779768, 0.016617441549897194, 0.017607884481549263, -0.004512020852416754, -0.01117000263184309, 0.004814655985683203, 0.0127106923609972, 0.008088622242212296, 0.008969016373157501, 0.023440496996045113, 0.05590503662824631, -0.010124534368515015, 0.002352303359657526, 0.04225892573595047, -0.018378229811787605, -0.035215768963098526, -0.05018247291445732, 0.019698821008205414, 0.023880694061517715, 0.014856653288006783, -0.022780202329158783, 0.011610199697315693, -0.005777587182819843, -0.03961774334311485, 0.006465395446866751, -0.018488280475139618, 0.0216797087341547, 0.0316941924393177, 0.01617724448442459, -0.011390101164579391, -0.02057921513915062, -0.02993340604007244, 0.007648425176739693, -0.022119905799627304, 0.01028960756957531, 0.00858384370803833, -0.00042300194036215544, 0.002090936293825507, 0.03059370070695877, 0.013315963558852673, -0.006630469113588333, -0.01859832927584648, 0.009189114905893803, 0.01441645622253418, -0.026521876454353333, 0.029273109510540962, 0.010949904099106789, -0.007015641778707504, 0.015626998618245125, -0.00619027204811573, 0.03565596789121628, 0.007923548109829426, -0.0034527962561696768, -0.005777587182819843, 0.019808871671557426, -0.0023798157926648855, 0.010729805566370487, 0.004016798920929432, -0.01716768741607666, -0.029713306576013565, 0.026631927117705345, 0.000997321680188179, -0.03543587028980255, 0.034555476158857346, -0.002173473360016942, -0.029053010046482086, -0.006052710581570864, 0.0032326977234333754, 0.02817261591553688, -0.01083985436707735, 0.004319434519857168, -0.02443094179034233, 0.023110348731279373, 0.016727490350604057, -0.009739361703395844, 0.019698821008205414, 0.028612812981009483, 0.040057938545942307, 0.0316941924393177, 0.013260938227176666, -0.009024041704833508, -0.01238054409623146, 0.009684337303042412, 0.05106286704540253, -0.015406900085508823, -0.013370987959206104, -0.01958877220749855, -0.011885322630405426, -0.00522734085097909, 0.016397343948483467, -0.027622370049357414, 0.023880694061517715, 0.006740518845617771, -0.026631927117705345, -0.022560102865099907, 0.007373301777988672, 0.011114977300167084, 0.03983784094452858, -0.017607884481549263, -0.0014375187456607819, 0.026962075382471085, 0.014526505023241043, 0.0008803942473605275, -0.008143646642565727, 0.006630469113588333, 0.04093833267688751, -0.032134391367435455, -0.020469166338443756, 0.015957146883010864, -0.03235448896884918, -0.03543587028980255, -0.00720822811126709, 0.029273109510540962, -0.04027803614735603, 0.0062177847139537334, -0.02300029993057251, 0.03807705268263817, 0.0002630865783430636, -0.00720822811126709, -0.01716768741607666, -0.004512020852416754, -0.007593400776386261, -0.012655667960643768, -0.014196357689797878, 0.009794386103749275, -0.0058050998486578465, 0.014196357689797878, -0.04820158705115318, 0.0034665523562580347, 0.03939764201641083, -0.024651039391756058, -0.016947589814662933, 0.0004539532819762826, -0.00929916463792324, 0.01408630795776844, 0.022780202329158783, -0.024540990591049194, 0.03719665855169296, -0.02443094179034233, -0.010729805566370487, 0.0007737840060144663, -0.02575153298676014, 0.010014484636485577, -0.0069055925123393536, -0.006575444713234901, -0.015406900085508823, -0.0012036639964208007, 0.03125399723649025, -0.001994643360376358, -0.008693893440067768, -0.019368674606084824, -0.003026355290785432, 0.017387786880135536, -0.005832612048834562, 0.009354189038276672, -0.007428326644003391, -0.0029025499243289232, 0.004456995986402035, 0.046220697462558746, 0.006960617378354073, 0.047981489449739456, -0.021569659933447838, -0.006878080312162638, -0.011555174365639687, -0.011225027032196522, -0.014636554755270481, -0.032794687896966934, -0.009904435835778713, 0.015076751820743084, -0.01815813221037388, 0.023770645260810852, 0.018708378076553345, -0.047541290521621704, 0.01238054409623146, 0.015516948886215687, 0.017717935144901276, 0.014306406490504742, -0.04908198118209839, -0.030373603105545044, 0.019698821008205414, -0.012105421163141727, -0.024651039391756058, 0.03191429376602173, -0.017607884481549263, 0.008033597841858864, 0.0031776730902493, 0.022560102865099907, 0.006025198381394148, 0.0015750803286209702, 0.0049247052520513535, -0.01408630795776844, 0.008418770506978035, 0.0018020570278167725, 0.001843325444497168, -0.01892847754061222, 0.044459909200668335, 0.00748335151001811, -0.0073182773776352406, 0.01496670302003622, -0.007923548109829426, -0.03389517962932587, 0.00619027204811573, -0.003934261854737997, -0.04468001052737236, -0.004291921854019165, -0.013811185024678707, 0.0174978356808424, -0.030153503641486168, -0.0031501606572419405, -0.02993340604007244, -0.0045670452527701855, -0.07131193578243256, -0.019698821008205414, -0.005667537916451693, 0.016947589814662933, 0.041378531605005264, 0.017607884481549263, 0.018378229811787605, -0.020139018073678017, -0.010894878767430782, 0.0019808870274573565, 0.006795543245971203, -0.05282365530729294, -0.013260938227176666, 0.02641182765364647, -0.01496670302003622, 0.016287293285131454, -0.015737047418951988, 0.012765716761350632, 0.0174978356808424, 0.025971630588173866, 0.010949904099106789, -0.027402272447943687, 0.007758474443107843, -0.025641482323408127, -0.022229954600334167, 0.03873734921216965, 0.02101941406726837, 0.032794687896966934, -0.024651039391756058, -0.03059370070695877, 0.0003903310571331531, 0.010784829966723919, -0.02949320711195469, -0.030373603105545044, 0.021129462867975235, -0.015957146883010864, 0.005557488650083542, 0.007593400776386261, -0.0003077940782532096, 0.03125399723649025, 0.006878080312162638, -0.013921234756708145, -0.007098178844898939, -0.00037141633220016956, 0.01061975583434105, 0.0006465395563282073, 0.02057921513915062, 0.00304011139087379, 0.008143646642565727, -0.02101941406726837, 0.0042093852534890175, 0.006575444713234901, -0.005034754518419504, 0.02300029993057251, -0.016947589814662933, -0.027952518314123154, 0.03191429376602173, 0.02575153298676014, -0.007813499309122562, -0.025531433522701263, 0.0013068352127447724, 0.026301778852939606, 0.005667537916451693, 0.014526505023241043, -0.013701136223971844, -0.011225027032196522, 0.0016576172783970833, -0.021349560469388962, -0.0033977716229856014, -0.01958877220749855, 0.024981187656521797, -0.007373301777988672, 0.03609616681933403, 0.045120205730199814, 0.00748335151001811, -0.06074720621109009, 0.0014031283790245652, 0.023440496996045113, -0.012270495295524597, -0.016397343948483467, -0.022560102865099907, 0.023440496996045113, 0.016067195683717728, -0.006740518845617771, 0.010014484636485577, -0.004401971120387316, -0.01287576649338007, -0.006933104712516069, -0.014746603555977345, -0.0174978356808424, -0.03323488309979439, -0.002035911660641432, 0.014526505023241043, 0.031033897772431374, -0.015626998618245125, 0.0058050998486578465, 0.009794386103749275, -0.007978573441505432, -0.0016920077614486217, 0.014636554755270481, -0.053043752908706665, -0.0012174202129244804, -0.01925862394273281, 0.01287576649338007, 0.040718235075473785, 0.011940347030758858, -0.013260938227176666, -0.01045468170195818, 0.029053010046482086, 0.023770645260810852, 0.015626998618245125, 0.013811185024678707, 0.0212395116686821, 0.002200985560193658, 0.018268181011080742, -0.01859832927584648, -0.028612812981009483, -0.02993340604007244, -0.008308720774948597, 0.011555174365639687, 0.015406900085508823, -0.03301478549838066, 0.002200985560193658, 0.008528819307684898, 0.03191429376602173, 0.02751232124865055, 0.004126848187297583, -0.001747032394632697, -0.008693893440067768, 0.01782798394560814, 0.021569659933447838, 0.005942661315202713, -0.010234583169221878, 0.006272809114307165, -0.010069509036839008, -0.016947589814662933, -0.006327833980321884, -0.0062177847139537334, -0.0072632525116205215, 0.0030813799239695072, -0.014526505023241043, -0.005585001315921545, 0.053704049438238144, 0.008528819307684898, -0.00011004928092006594, -0.008198671974241734, -0.0127106923609972, 0.005557488650083542, 0.001169273629784584, 0.02267015166580677, -0.013866209425032139, -0.012490593828260899, 0.025311335921287537, 0.009574287571012974, -0.01061975583434105, 0.002434840425848961, 0.020359117537736893, 0.01441645622253418, 0.015737047418951988, 0.043139319866895676, 0.014031283557415009, 0.04776138812303543, -0.0003198307240381837, -0.013426012359559536, -0.010344632901251316, 0.01100492849946022, -0.0063553461804986, 0.011775272898375988, 0.0035490894224494696, 0.025971630588173866, -0.009574287571012974, -0.024871138855814934, -0.022780202329158783, 0.01254561822861433, 0.0391775444149971, 0.01650739274919033, -0.005667537916451693, -0.010069509036839008, -0.004456995986402035, -0.008253696374595165, -0.016067195683717728, -0.013591086491942406, 0.045120205730199814, 0.0024898650590330362, -0.007153203245252371, 0.026852024719119072, -0.040718235075473785, -0.00946423877030611, -0.006740518845617771, -0.023880694061517715, -0.003507820889353752, 0.006768031045794487, -0.026301778852939606, -0.030153503641486168, -0.013260938227176666, 0.001513177645392716, 0.031033897772431374, 0.03741675615310669, -0.02993340604007244, -0.006685493979603052, 0.016287293285131454, 0.005199828650802374, -0.026631927117705345, -0.018048083409667015, -0.02267015166580677, 0.017607884481549263, 0.0005227340734563768, -0.02575153298676014, -0.0026549389585852623, -0.022119905799627304, 0.013701136223971844, -0.03235448896884918, 0.040718235075473785, -0.02674197591841221, 0.026631927117705345, -0.005970173515379429, 0.03301478549838066, 0.007373301777988672, 0.013205913826823235, 0.0028200128581374884, 0.05810602009296417, -0.018378229811787605, 0.02784246951341629, -0.01496670302003622, 0.03301478549838066, -0.019478723406791687, 0.002159717259928584, -0.007098178844898939, -0.031033897772431374, -0.023110348731279373, -0.011885322630405426, -0.005915149115025997, 0.013536062091588974, -0.0058050998486578465, 0.017717935144901276, 0.0021184487268328667, -0.029273109510540962, 0.013811185024678707, 0.008308720774948597, -0.0006327833980321884, 0.01441645622253418, 0.0073182773776352406, 0.0316941924393177, -0.030373603105545044, -0.024981187656521797, 0.04093833267688751, 0.013921234756708145, 0.013150889426469803, -0.01892847754061222, 0.0053373901173472404, -0.011775272898375988, 0.01216044556349516, 0.010509707033634186, 0.021459611132740974, 0.053043752908706665, -0.02608167938888073, 0.014636554755270481, 0.02883291244506836, 0.0014787872787564993, 0.03191429376602173, 0.02784246951341629, -0.02949320711195469, 0.014526505023241043, -0.01028960756957531, 0.005722562782466412, -0.04203882813453674, -0.017717935144901276, -0.029273109510540962, -0.03323488309979439, -0.013646110892295837, 0.02641182765364647, -0.01617724448442459, 0.015406900085508823, -0.005970173515379429, 0.007813499309122562, -0.003507820889353752, -0.005529976449906826, 0.016397343948483467, -0.0032877223566174507, -0.02949320711195469, 0.020359117537736893, 0.017277738079428673, -0.01496670302003622, 0.005474952049553394, 0.019038526341319084, 0.014526505023241043, -0.03609616681933403, 0.021349560469388962, 0.001760788494721055, 0.046880993992090225, -0.006768031045794487, -0.01496670302003622, -0.01496670302003622, -0.0174978356808424, -0.008143646642565727, 0.038957446813583374, 0.007813499309122562, 0.0022972787264734507, -0.011500149965286255, -0.0004040872154291719, 0.0003662577655632049, 0.02509123645722866, 0.0708717405796051, -0.007428326644003391, 0.029053010046482086, 0.015737047418951988, -0.010729805566370487, -0.01584709621965885, 0.01782798394560814, -0.013811185024678707, 0.006492907647043467, -0.04534030333161354, -0.025641482323408127, -0.00704315397888422, -0.008803942240774632, -0.00024761087843216956, -0.0007806621142663062, 0.030373603105545044, -0.012765716761350632, 0.013976259157061577, -0.022450054064393044, -0.007923548109829426, 0.03367508202791214, 0.018488280475139618, 0.004484508186578751, 0.044459909200668335, 0.00037141633220016956, -0.0072632525116205215, 0.006465395446866751, 0.01083985436707735, -0.023110348731279373, -0.005062267184257507, -0.010344632901251316, -0.02751232124865055, 0.021129462867975235, -0.000254488957580179, 0.022450054064393044, 0.02575153298676014, -0.00858384370803833, -0.043139319866895676, 0.03257458657026291, 0.028392715379595757, 0.012655667960643768, 0.02410079352557659, 0.020139018073678017, -0.022780202329158783, -0.016287293285131454, -0.03125399723649025, 0.017938032746315002, -0.003851725021377206, -0.011940347030758858, -0.034115277230739594, -0.006685493979603052, -0.015406900085508823, -0.0057500749826431274, 0.00720822811126709, -0.018378229811787605, 0.001176151679828763, 0.029053010046482086, 0.020139018073678017, 0.01650739274919033, 0.013646110892295837, 0.023550545796751976, 0.0008425648557022214, 0.03477557376027107, -0.05106286704540253, -0.007373301777988672, -0.016727490350604057, 0.02200985699892044, 0.003604114055633545, -0.00619027204811573, -0.002613670425489545, 0.005640025716274977, 0.0072632525116205215, -0.01028960756957531, -0.011500149965286255, 0.034555476158857346, -0.05326385423541069, -0.008253696374595165, -0.030153503641486168, 0.032794687896966934, 0.020359117537736893, 0.01815813221037388, 0.00022009856184013188, -0.0006155881565064192, -0.03543587028980255, 0.033454980701208115, 0.03719665855169296, -0.01925862394273281, -0.016287293285131454, 0.026962075382471085, -0.030153503641486168, 0.0010248339967802167, -0.019478723406791687, 0.006410370580852032, 0.012600642628967762, 0.006025198381394148, 0.017607884481549263, -0.024651039391756058, -0.015076751820743084, 0.007648425176739693, -0.029273109510540962, -0.013536062091588974, 0.0010661024134606123, -0.024981187656521797, 0.029713306576013565, -0.006300321314483881, -0.013260938227176666, -0.007703450042754412, 0.01083985436707735, 0.015957146883010864, 0.02366059646010399, -0.024651039391756058, 0.020689265802502632, -0.02432089112699032, -0.003989286720752716, -0.0508427694439888, 0.007648425176739693, -0.026962075382471085, 0.005309877917170525, 0.0316941924393177, 0.020028969272971153, 0.0316941924393177, 0.015957146883010864, -0.0032877223566174507, -0.0016851297114044428, 0.01958877220749855, -0.014196357689797878, 0.034555476158857346, -0.011830298230051994, 0.013866209425032139, -0.010564731433987617, 0.0027237196918576956, -0.004897193051874638, -0.016287293285131454, 0.04115843027830124, 0.006685493979603052, 0.0037141633220016956, -0.009739361703395844, -0.01100492849946022, 0.009684337303042412, -0.004704606719315052, 0.027292221784591675, 0.006988129578530788, -0.013426012359559536, -0.006245296914130449, 0.011720248498022556, 0.04357951506972313, 0.005612513516098261, -0.008088622242212296, 0.004071823321282864, 0.011555174365639687, -0.0030676238238811493, 0.021569659933447838, 0.04842168465256691, 0.008363745175302029, -0.024651039391756058, 0.02443094179034233, -0.04776138812303543, 0.06074720621109009, 0.045120205730199814, -0.021569659933447838, 0.0127106923609972, 0.011225027032196522, -0.008638869039714336, 0.01925862394273281, 0.022890251129865646, -0.01991892047226429, -0.02267015166580677, 0.03609616681933403];

  3. Save and close the file.

2

Connect to your cluster using mongosh.

Open mongosh in a terminal window and connect to your cluster. For detailed instructions on connecting, see Connect to a Cluster via mongosh.

3

Use the sample_mflix database.

Run the following command at mongosh prompt:

use sample_mflix
switched to db sample_mflix
4

Load the embeddings to use in the query.

Load the query-embeddings.js file into mongosh.

load('<path-to-file>/query-embeddings.js');
5

Run the following MongoDB Search queries against the embedded_movies collection.

1vectorWeight = 1;  
2vectorCutoff = 0.7; 
3numResults = 20;
4numCandidates = numResults * 10;  
5
6// Execute the vector search  
7vectorResults = db.embedded_movies.aggregate([  
8    {  
9        $vectorSearch: {  
10            "index": "hybrid-vector-search", 
11            "path": "plot_embedding_voyage_4_large",
12            "queryVector": CHARMING_ANIMAL_EMBEDDING,
13            "numCandidates": numCandidates,   
14            "limit": numResults  
15        }  
16    },  
17    {  
18        $project: {  
19            "_id": 1,  
20            "title": 1,  
21            "fullplot": 1,
22            "vectorScore": { $meta: "vectorSearchScore" }  
23        }  
24    },  
25    {  
26        $match: {  
27            "vectorScore": { $gte: vectorCutoff }  
28        }  
29    }  
30]).toArray();  
31
32// Create a map for fast lookup  
33vectorScoresMap = {};  
34vectorResults.forEach(result => {  
35    vectorScoresMap[result._id] = result.vectorScore * vectorWeight;  
36});  
37
38// Execute the text search with boosting based just on ID  
39textResults = db.embedded_movies.aggregate([  
40    {  
41        $search: {  
42            "index": "hybrid-full-text-search",  
43            "compound": {  
44                "should": [  
45                    {  
46                        "text": {  
47                            "query": "charming animal",  
48                            "path": "fullplot",
49                            "fuzzy": {}
50                        }  
51                    },  
52                    ...Object.keys(vectorScoresMap).map(id => ({  
53                        "equals": {  
54                            "path": "_id",  
55                            "value": id,  
56                            "score": { boost: { value: vectorScoresMap[id] } }  
57                        }  
58                    }))  
59                ]  
60            }  
61        }  
62    },  
63    {
64        $limit: 20
65    },
66    {  
67        $project: {  
68            "_id": 1,  
69            "title": 1,  
70            "fullplot": 1,
71            "score": { $meta: "searchScore" }  
72        }  
73    }  
74]).toArray();

About the Query

The sample query boosts the $search results based on $vectorSearch result scores to return the most relevant documents. For semantic boosting, you perform the following steps:

  1. Run the $vectorSearch query.

    Use an over request value to specify the number of candidates more than the specified limit to consider. The over request allows you to accomplish the following:

    • Include a large number and broad pool of documents to consider in the initial search.

    • Refine scoring computation in subsequent stages.

    The query uses the following calculation to determine the number of candidates to consider during the vector search:

    numCandidates = limit * over request

    The results are filtered based on a minimum score threshold (vectorCutoff).

  2. Scale the vector search score to adjust the influence of the vector search results on the scoring and ranking of documents.

    Use the vectorWeight parameter to set the weight and multiply the vector search scores by the vectorWeight. This controls the importance of the vector search results and ensures that only the most semantically relevant documents are considered.

  3. Run the $search query and boost the score of the documents that match the results from the vector search.

Query Variables

Variables used in the query.

The sample query defines the following variables:

  • vectorWeight to add weight to the vector search score, with a lower number providing higher weight.

  • vectorCutoff to set the minimum vector score for result to be boosted.

  • numResults to specify the desired number of search results

  • numCandidates to calculate the number of candidates to consider during the initial search. It calculates the number of candidates by using the following formula:

    numCandidates = numResults x overRequestFactor
Query Stages

Aggregations pipeline stages used in the query.

The sample query uses the following pipeline stages to perform a semantic search on the collection and retrieve the reciprocal rank of the documents in the results:

Searches the plot_embedding_voyage_4_large field for the string charming animal, specified as vector embeddings in the queryVector field of the query. The query uses the voyage-4-large embedding model from Voyage AI, which is the same model used for the embeddings in the plot_embedding_voyage_4_large field.

Searches for movies that contain the term charming animal in the fullplot field. For the documents in the results that match the _id from the vector search results, the query boosts the score by the adjusted vector search score for the document.

Reciprocal Rank Fusion Query

You can run a query that retrieves the sorted search results from the semantic search and the full-text search, and assigns a reciprocal rank score to the documents in the results based on their position in the results array. To learn more about the reciprocal rank fusion technique and how to run a reciprocal rank fusion query, see Perform Hybrid Vector and Full-Text Search.

Relative Score Fusion Query

You can run a query that retrieves the scored search results from the semantic and full-text searches and applies the sigmoid expression to normalize the score before combining the results from the pipelines into a single aggregated score for each document. To learn more about the score fusion technique and how to run a score fusion query, see Perform Hybrid Vector and Full-Text Search.

Back
Customize Score
Next
Native Reranking
Rate this page

On this page