For AI agents: a documentation index is available at https://www.mongodb.com/docs/llms.txt — markdown versions of all pages are available by appending .md to any URL path.
Docs Menu
Docs Home
/ /
Improve Accuracy
Docs Home
/
Development
/
Search
/
Improve Accuracy
Docs Home
/
Development
/
Search
/
Improve Accuracy

How to Perform Hybrid Search

You can combine MongoDB Vector Search and MongoDB Search queries into a hybrid search for unified results.

This tutorial demonstrates how to run a hybrid search on the sample_mflix.embedded_movies collection, which contains details about movies. Specifically, this tutorial takes you through the following steps:

  1. Create a MongoDB Vector Search index on the plot_embedding_voyage_4_large field. This field contains vector embeddings (of 2048 dimensions created using Voyage AI's voyage-4-large embedding model) that represent the summary of a movie's plot.

  2. Create a MongoDB Search index on the sample_mflix.embedded_movies collection.

  3. Run a query to combine the results from a $vectorSearch query and a $search query by using the following techniques:

    • Semantic boosting

    • Reciprocal rank fusion

    • Score Fusion

Why Hybrid Search?

A hybrid search is an aggregation of different search methods, such as a full-text and semantic search, for the same query criteria. While full-text is effective in finding exact matches for query terms, semantic search provides the added benefit of identifying semantically similar documents even if the documents don't contain the exact query term. This ensures that synonymous and contextually similar matches are also included in the combined results of both methods of search.

Conversely, if you have tokens for proper nouns or specific keywords in your dataset that you don't expect to be considered in the training of an embedding model in the same context that they are used in your dataset, your vector search might benefit from being combined with a full-text search.

You can also set weights for each method of search per query. Based on whether full-text or semantic search results are most relevant and appropriate for a query, you can increase the weight for that search method per query.

You can apply the following techniques to achieve a hybrid search:

Technique

Semantic Boosting

Reciprocal Rank Fusion

Relative Score Fusion

Purpose

Improves the accuracy of the full-text search results by boosting documents that match vector search. Useful when retrieving very large result sets.

Combines results from different search methods into a single ranked result set by calculating the reciprocal rank of documents in the results.

Combines results from different search methods into a single ranked result set by calculating the score of documents in the results.

Ranking Method

  • Run $vectorSearch to retrieves semantically similar results.

  • Run $search to retrieve full-text results while boosting the documents that were returned by $vectorSearch.

  • Retrieves $vectorSearch and $search results.

  • Merges both sets of results using the rank of each document in the respective result sets.

  • Retrieves $vectorSearch and $search results.

  • Merges both sets of results using the score of each document in the respective result sets.

Scoring Method

Boosts the full-text results score for documents that match the results from the vector search, by the weighted vector search score.

Calculates the score by using the following formula:

1.0/{document position in the results + constant value}

Then, for each method of search, applies different weights (w) to give more importance to that method of search. For each document, the weighted reciprocal rank is calculated by multiplying the weight by the reciprocal rank of the document.

weighted_reciprocal_rank = w x reciprocal_rank

Calculates the score by using the following formula:

score = w1 x score1 + w2 x score2

Search Features

  • Supports faceting to group data into categories.

  • Supports paginating results using searchAfter and searchBefore.

Supports further processing of results in subsequent aggregation stages such as $group.

Supports further processing of results in subsequent aggregation stages such as $group.

Output

Single result set that contains documents from $search

Single result set that contains documents from both search methods.

Single result set that contains documents from both search methods.

Prerequisites

Before you begin, you must have the following:

  • An Atlas cluster with MongoDB version v8.0 or higher.

    Note

    Ensure that your Atlas cluster has enough memory to store both MongoDB Search and MongoDB Vector Search indexes and run performant queries.

  • The sample_mflix data loaded into your Atlas cluster.

  • mongosh to run queries on your Atlas cluster:

  • Project Data Access Admin access to the project to create MongoDB Vector Search and MongoDB Search indexes.

Create the MongoDB Vector Search and MongoDB Search Indexes

This section demonstrates how to create the following indexes on the fields in the sample_mflix.embedded_movies collection:

  • A MongoDB Vector Search index on the plot_embedding_voyage_4_large field for running vector queries against that field.

  • A MongoDB Search index that dynamically indexes all dynamically indexable fields for running full-text search against those fields.

1

Connect to the cluster using mongosh.

To learn more, see Connect to a Cluster via mongosh.

2

Switch to the database that contains the collection for which you want to create the index.

use sample_mflix
switched to db sample_mflix
3

Define the MongoDB Vector Search index.

Run the following command. This index definition indexes the plot_embedding_voyage_4_large field as the MongoDB Vector Search field when querying the collection.

1db.embedded_movies.createSearchIndex(
2  "hybrid-vector-search", 
3  "vectorSearch", 
4  {
5    "fields": [
6      {
7        "type": "vector",
8        "path": "plot_embedding_voyage_4_large",
9        "numDimensions": 2048,
10        "similarity": "dotProduct"
11      }
12    ]
13  }
14);
4

Define the MongoDB Search index.

The following index definition automatically indexes all the dynamically indexable fields in the collection.

1db.embedded_movies.createSearchIndex(
2  "hybrid-full-text-search", 
3  "search", 
4  {
5    "mappings": {
6      "dynamic": true
7    }
8  }
9);

Run the Hybrid Search Queries

This section demonstrates how to query the data in the sample_mflix.embedded_movies collection for charming animal in the plot_embedding_voyage_4_large and fullplot fields by using the $vectorSearch and $search pipeline stages, respectively. The query enhances relevancy of $search queries by boosting the full-text scores using $vectorSearch scores for the same documents.

Semantic Boosting Query

1

Create the file for the embeddings.

  1. Create a file named query-embeddings.js.

    touch query-embeddings.js

  2. Copy and paste the following embeddings into the file.

    CHARMING_ANIMAL_EMBEDDING=[-0.02300029993057251, -0.040718235075473785, 0.026521876454353333, -0.007813499309122562, 0.01958877220749855, -0.032134391367435455, 0.015186801552772522, 0.023990744724869728, -0.0029025499243289232, -0.0019671309273689985, 0.01254561822861433, -0.016287293285131454, -0.029053010046482086, 0.01408630795776844, -0.00891399197280407, 0.01958877220749855, -0.0508427694439888, -0.015737047418951988, 0.012215470895171165, -0.0036316262558102608, 0.0033152345567941666, 0.0033840155228972435, -0.00748335151001811, 0.002737476024776697, 0.03257458657026291, -0.01892847754061222, -0.012050396762788296, 0.02432089112699032, 0.028392715379595757, 0.033454980701208115, 0.005144803784787655, 0.0072632525116205215, 0.0212395116686821, -0.01683754101395607, -0.002751232124865055, 0.030813800171017647, -0.025861581787467003, 0.00748335151001811, -0.0032326977234333754, -0.030813800171017647, 0.006768031045794487, 0.024651039391756058, -0.03565596789121628, -0.02575153298676014, -0.005970173515379429, 0.02575153298676014, -0.08011587709188461, 0.0022284979932010174, -0.03257458657026291, 0.00522734085097909, -0.0028062567580491304, 0.011059952899813652, 0.0061627598479390144, 0.021349560469388962, -0.0061627598479390144, 0.02200985699892044, 0.006988129578530788, -0.006850568111985922, 0.010784829966723919, -0.0035490894224494696, -0.007538375910371542, 0.0023110350593924522, -0.031033897772431374, -0.04291922226548195, -0.03389517962932587, -0.004181872587651014, -0.02024906873703003, 0.003769187955185771, -0.01650739274919033, -0.0031226484570652246, -0.0026686950586736202, 0.03389517962932587, 0.020689265802502632, -0.025531433522701263, 0.02024906873703003, 0.03125399723649025, 0.006602956913411617, -0.01441645622253418, -0.04974227771162987, 0.005887636449187994, -0.013150889426469803, 0.007593400776386261, 0.00352157698944211, -0.014196357689797878, -0.009849410504102707, -0.019698821008205414, -0.01441645622253418, 0.0018708378775045276, -0.007703450042754412, 0.0020771801937371492, -0.03565596789121628, 0.027402272447943687, 0.007593400776386261, 0.005695050582289696, 0.03389517962932587, 0.014196357689797878, -0.002682451391592622, 0.030373603105545044, -0.01061975583434105, -0.017387786880135536, -0.00522734085097909, -0.013205913826823235, 0.007813499309122562, 0.02509123645722866, 0.00304011139087379, -0.04225892573595047, -0.011500149965286255, 0.005887636449187994, 0.0016163488617166877, -0.03477557376027107, 0.04423981159925461, -0.005144803784787655, -0.004401971120387316, -0.028062567114830017, 0.0087489178404212, 0.0029850867576897144, -0.0062177847139537334, -0.012215470895171165, -0.001430640695616603, 0.017277738079428673, -0.007813499309122562, -0.03191429376602173, 0.033454980701208115, 0.018818426877260208, 0.0035766016226261854, -0.030373603105545044, -0.056125134229660034, -0.01815813221037388, 0.0031501606572419405, 0.013756160624325275, -0.03785695508122444, -0.011775272898375988, 0.04908198118209839, 0.031474094837903976, -0.0004126848070882261, 0.027732418850064278, 0.01716768741607666, 0.005640025716274977, -0.013646110892295837, -0.010124534368515015, -0.005007242318242788, -0.02300029993057251, 0.027622370049357414, 0.015737047418951988, -0.03323488309979439, -0.00018312888278160244, -0.010509707033634186, -0.0024898650590330362, -0.014746603555977345, -0.004704606719315052, 0.03191429376602173, 0.02443094179034233, -0.0013756160624325275, -0.0046220701187849045, -0.049962375313043594, -0.01782798394560814, 0.0003920505696441978, -0.01892847754061222, 0.0012311763130128384, 0.041378531605005264, -0.041378531605005264, -0.02090936340391636, -0.010234583169221878, -0.008418770506978035, 0.03477557376027107, 0.009684337303042412, 0.05194326117634773, -0.010344632901251316, -0.047541290521621704, 0.010509707033634186, -0.03477557376027107, -0.023440496996045113, -0.007153203245252371, 0.002737476024776697, 0.020689265802502632, 0.020689265802502632, 0.03301478549838066, 0.012490593828260899, -0.024210842326283455, -0.0216797087341547, 0.005062267184257507, 0.0032877223566174507, -0.02949320711195469, -0.00232479115948081, -0.023990744724869728, 0.04534030333161354, 0.010124534368515015, 0.015516948886215687, 0.04490010812878609, -0.0007084422395564616, -0.04842168465256691, -0.025531433522701263, 0.0216797087341547, -0.010399657301604748, 0.049962375313043594, -0.02024906873703003, 0.03477557376027107, 0.008803942240774632, 0.02057921513915062, -0.01408630795776844, -0.06074720621109009, -0.010014484636485577, 0.034115277230739594, -0.017387786880135536, -0.02410079352557659, -0.01100492849946022, -0.02575153298676014, -0.026521876454353333, -0.03741675615310669, 0.011610199697315693, -0.045560404658317566, 0.0005811977898702025, 0.016067195683717728, 0.008473794907331467, -0.02024906873703003, -0.03961774334311485, 0.00748335151001811, 0.02784246951341629, 0.021349560469388962, 0.02333044819533825, 0.009574287571012974, 0.0006431004730984569, -0.035876065492630005, 0.002022155560553074, 0.019368674606084824, -0.004181872587651014, -0.011335075832903385, -0.02883291244506836, -0.000839125772472471, 0.006713006179779768, -0.00030263554072007537, -0.0025724018923938274, -0.013811185024678707, 0.001843325444497168, -0.013701136223971844, -0.017277738079428673, -0.006245296914130449, -0.006933104712516069, -0.014636554755270481, 0.0033977716229856014, 0.004044311121106148, -0.01584709621965885, 0.00858384370803833, -0.0042093852534890175, 0.0005124169983901083, -0.013315963558852673, -0.03301478549838066, -0.011059952899813652, -0.02057921513915062, 0.0010385900968685746, -0.002421084325760603, 0.00043503858614712954, 0.02443094179034233, 0.009794386103749275, -0.002751232124865055, 0.006382858380675316, 0.018488280475139618, 0.03565596789121628, -0.021789757534861565, 0.011830298230051994, -0.021569659933447838, -0.00034906258224509656, -0.032134391367435455, 0.015406900085508823, -0.02432089112699032, -0.004181872587651014, 0.010509707033634186, 0.020689265802502632, 0.04468001052737236, -0.018048083409667015, -0.00014701895997859538, -0.036536362022161484, 0.02993340604007244, 0.022890251129865646, -0.024210842326283455, 0.02366059646010399, 0.0005777587648481131, 0.011775272898375988, -0.0349956713616848, 0.05458444356918335, 0.004842168651521206, 0.013646110892295837, 0.022780202329158783, -0.006520419847220182, -0.008803942240774632, -0.0035766016226261854, 0.052383460104465485, -0.011940347030758858, 0.0027925006579607725, 0.011280051432549953, 0.029053010046482086, 0.015406900085508823, 0.009959460236132145, -0.0011899078963324428, 0.035876065492630005, -0.001671373494900763, 0.006575444713234901, -0.014306406490504742, 0.001011077780276537, -0.020689265802502632, -0.021129462867975235, -0.004704606719315052, 0.003342746989801526, -0.005832612048834562, -0.0025724018923938274, -0.003769187955185771, -0.005447439383715391, 0.027952518314123154, 0.011830298230051994, 0.03961774334311485, 0.0550246424973011, 0.0031364045571535826, 0.0012311763130128384, -0.002764988224953413, -0.05150306597352028, -0.02608167938888073, 0.015296850353479385, -0.0020634240936487913, -0.03675645962357521, 0.026962075382471085, 0.023770645260810852, -0.011940347030758858, -0.05898641422390938, -0.03785695508122444, -0.003026355290785432, 0.0021322048269212246, 0.02509123645722866, -0.01925862394273281, 0.03961774334311485, 0.02333044819533825, -0.00031123313237912953, 0.06206779554486275, 0.00010102180385729298, -0.01683754101395607, -0.02751232124865055, 0.03851724788546562, 0.0073182773776352406, 0.02641182765364647, -0.042699120938777924, -0.008969016373157501, -0.01287576649338007, 0.018268181011080742, 0.01991892047226429, -0.002916306024417281, 0.032794687896966934, -0.03873734921216965, 0.023550545796751976, -0.03191429376602173, -0.0072632525116205215, 0.0022972787264734507, 0.023440496996045113, -0.01441645622253418, 0.0035766016226261854, 0.016287293285131454, 0.021569659933447838, -0.0004505142569541931, -0.0028062567580491304, -0.004099335987120867, 0.015516948886215687, 0.02024906873703003, 0.004787143785506487, 0.03477557376027107, -0.009244140237569809, 0.026852024719119072, -0.002778744325041771, 0.003741675755009055, 0.015296850353479385, 0.029053010046482086, -0.020139018073678017, 0.006492907647043467, -0.02575153298676014, 0.03983784094452858, 0.0038242125883698463, 0.010344632901251316, -0.012490593828260899, -0.02333044819533825, -0.019808871671557426, -0.04490010812878609, 0.013205913826823235, -0.037636853754520416, 0.00891399197280407, 0.0011555175296962261, 0.022450054064393044, 0.04225892573595047, 0.0018158131279051304, 0.004869680851697922, 0.02949320711195469, 0.01496670302003622, -0.013150889426469803, -0.04776138812303543, -0.020359117537736893, -0.011665224097669125, -0.00464958231896162, -0.016287293285131454, -0.024651039391756058, -0.005887636449187994, -0.0020634240936487913, 0.0014581530122086406, -0.020028969272971153, -0.002861281391233206, 0.0072632525116205215, -0.022890251129865646, -0.027292221784591675, 0.017938032746315002, 0.03125399723649025, -0.012600642628967762, -0.02333044819533825, 0.03477557376027107, -0.027072124183177948, 0.022780202329158783, -0.010509707033634186, 0.02784246951341629, -0.02057921513915062, -0.010949904099106789, -0.011830298230051994, -0.023550545796751976, 0.005364902783185244, 0.02476108819246292, 0.0002820013032760471, 0.0036591386888176203, 0.04423981159925461, 0.022780202329158783, 0.047981489449739456, -0.02200985699892044, 0.009904435835778713, 0.0036316262558102608, -0.007813499309122562, 0.02608167938888073, 0.014306406490504742, 0.011114977300167084, -0.040057938545942307, 0.01782798394560814, -0.022450054064393044, 0.043799616396427155, 0.0212395116686821, -0.022119905799627304, -0.015626998618245125, 0.013811185024678707, 0.026852024719119072, 0.0029025499243289232, -0.01782798394560814, 0.005777587182819843, 0.03851724788546562, -0.047981489449739456, 0.011610199697315693, 0.037636853754520416, 0.01408630795776844, -0.026191730052232742, 0.025311335921287537, 0.04181872680783272, 0.027402272447943687, -0.004512020852416754, -0.044459909200668335, 0.004236897453665733, -0.022229954600334167, -0.005309877917170525, 0.005254853516817093, 0.002517377259209752, 0.016067195683717728, 0.02476108819246292, -0.001341225695796311, -0.001341225695796311, 0.023440496996045113, -0.014746603555977345, 0.00748335151001811, 0.03389517962932587, -0.014031283557415009, -0.04886188358068466, -0.007813499309122562, -0.03389517962932587, 0.034115277230739594, -0.00522734085097909, 0.03807705268263817, -0.013095865026116371, -0.00619027204811573, 0.025641482323408127, 0.023770645260810852, -0.02751232124865055, 0.013315963558852673, 0.030373603105545044, -0.01958877220749855, 0.007923548109829426, -0.016727490350604057, 0.01782798394560814, -0.008693893440067768, -0.01496670302003622, 0.020139018073678017, -0.007703450042754412, -0.010069509036839008, 0.006602956913411617, 0.05216335877776146, 0.022450054064393044, 0.017607884481549263, -0.014526505023241043, -0.020689265802502632, 0.003961774054914713, 0.0020496679935604334, 0.013150889426469803, -0.035215768963098526, -0.024651039391756058, -0.02432089112699032, -0.013811185024678707, 0.031474094837903976, -0.034555476158857346, 0.006933104712516069, 0.038957446813583374, 0.013811185024678707, -0.008803942240774632, -0.02993340604007244, 0.01254561822861433, 0.010894878767430782, -0.011665224097669125, -0.0005674416315741837, -0.02101941406726837, 0.02883291244506836, -0.025201285257935524, 0.0007978572975844145, 0.003934261854737997, -0.004484508186578751, -0.02883291244506836, -0.02883291244506836, 0.0014031283790245652, -0.008803942240774632, -0.01287576649338007, -0.0010661024134606123, 0.025971630588173866, -0.024871138855814934, 0.020799314603209496, 0.010124534368515015, -0.03609616681933403, -0.026191730052232742, -0.015737047418951988, -0.003205185290426016, 0.0174978356808424, 0.03565596789121628, 0.02641182765364647, -0.006547932513058186, -0.00720822811126709, -0.005777587182819843, 0.007098178844898939, 0.03257458657026291, -0.020799314603209496, 0.0073182773776352406, -0.0216797087341547, 0.010949904099106789, -0.0010661024134606123, 0.00522734085097909, 0.011500149965286255, 0.0254213847219944, 0.008473794907331467, 0.01238054409623146, 0.011390101164579391, 0.013811185024678707, 0.0033014784567058086, -0.009684337303042412, -0.00453953305259347, 0.0012999571627005935, 0.008253696374595165, -0.0063553461804986, 0.0035766016226261854, 0.015737047418951988, -0.024540990591049194, 0.016947589814662933, 0.016397343948483467, -0.005144803784787655, 0.013260938227176666, 0.024981187656521797, -0.0073182773776352406, -0.038297150284051895, -0.016067195683717728, -0.006410370580852032, -0.011225027032196522, 0.012820741161704063, -0.009189114905893803, -0.01925862394273281, -0.02993340604007244, 0.012105421163141727, -0.0254213847219944, -0.006575444713234901, -0.02817261591553688, -0.025861581787467003, -0.012655667960643768, 0.034115277230739594, -0.03059370070695877, -0.00748335151001811, -0.006382858380675316, -0.005832612048834562, -0.015186801552772522, -0.01408630795776844, 0.029053010046482086, -0.026962075382471085, 0.012105421163141727, -0.056125134229660034, -0.004759631585329771, 0.0023798157926648855, 0.0016094708116725087, -0.01441645622253418, 0.0783550888299942, 0.026191730052232742, -0.013646110892295837, -0.04357951506972313, -0.019808871671557426, -0.0011486393632367253, -0.0038792372215539217, -0.017607884481549263, 0.012215470895171165, 0.03675645962357521, 0.044459909200668335, 0.017277738079428673, -0.029053010046482086, 0.0026962074916809797, 0.01045468170195818, -0.023110348731279373, 0.008363745175302029, -0.02090936340391636, 0.019698821008205414, -0.018378229811787605, -0.016727490350604057, 0.030813800171017647, 0.03301478549838066, -0.055464837700128555, 0.031474094837903976, 0.022229954600334167, -0.019478723406791687, -0.015186801552772522, 0.02333044819533825, 0.006850568111985922, 0.0024623526260256767, 0.03301478549838066, -0.012490593828260899, -0.017938032746315002, 0.0010454681469127536, -0.009134090505540371, -0.054144248366355896, 0.02333044819533825, 0.03477557376027107, -0.026631927117705345, -0.02674197591841221, -0.005062267184257507, -0.010179558768868446, -0.0005708806565962732, 0.01408630795776844, 0.011114977300167084, 0.01859832927584648, -0.0018708378775045276, 0.0016369831282645464, -0.01083985436707735, -0.019478723406791687, 0.022780202329158783, -0.004126848187297583, 0.032134391367435455, 0.02024906873703003, 0.007813499309122562, 0.00018656792235560715, 0.01859832927584648, -0.004512020852416754, 0.004126848187297583, -0.015626998618245125, -0.018268181011080742, -0.007098178844898939, -0.015626998618245125, -0.01815813221037388, 0.013921234756708145, 0.013811185024678707, 0.016617441549897194, 0.03543587028980255, 0.02024906873703003, 0.0010661024134606123, -0.015076751820743084, 0.003769187955185771, -0.011940347030758858, 0.0061627598479390144, -0.04401971399784088, -0.005640025716274977, -0.0009560531470924616, -0.023440496996045113, 0.03873734921216965, 0.005062267184257507, 0.01716768741607666, 0.001760788494721055, 0.002256010426208377, 0.013095865026116371, 0.011500149965286255, 0.019038526341319084, -0.006382858380675316, -0.0433594174683094, -0.0072632525116205215, -0.022119905799627304, 0.036536362022161484, 0.019698821008205414, 0.0018845939775928855, -0.029053010046482086, -0.004787143785506487, -0.02883291244506836, 0.005942661315202713, 0.045120205730199814, 0.0036316262558102608, 0.011665224097669125, 0.008253696374595165, 0.02509123645722866, 0.03323488309979439, 0.02718217298388481, -0.013040839694440365, 0.01782798394560814, 0.029053010046482086, -0.011280051432549953, 0.01216044556349516, -0.014196357689797878, 0.005640025716274977, 0.03059370070695877, -0.010124534368515015, -0.000997321680188179, -0.0004883437068201602, 0.004787143785506487, 0.009354189038276672, -0.007015641778707504, 0.009849410504102707, -0.02410079352557659, -0.0062177847139537334, -0.0031364045571535826, 0.004952217917889357, 0.020469166338443756, -0.029273109510540962, -0.01782798394560814, 0.004787143785506487, -0.024540990591049194, -0.012435569427907467, 0.017057638615369797, -0.0037967003881931305, 0.02267015166580677, -0.011335075832903385, 0.002448596525937319, 0.045560404658317566, 0.03125399723649025, 0.022229954600334167, 0.05898641422390938, 0.028392715379595757, 0.004044311121106148, -0.04181872680783272, 0.000254488957580179, 0.01254561822861433, -0.015737047418951988, -0.01496670302003622, 0.01925862394273281, 0.049302078783512115, -0.012325519695878029, -0.0017263981280848384, -0.0008425648557022214, 0.019038526341319084, -0.0073182773776352406, 0.018268181011080742, 0.030153503641486168, 0.055464837700128555, 0.04490010812878609, 0.019038526341319084, 0.04225892573595047, -0.022119905799627304, -0.015296850353479385, -0.0009285408305004239, -0.03741675615310669, -0.03961774334311485, 0.005254853516817093, 0.033454980701208115, 0.006272809114307165, 0.010894878767430782, -0.010069509036839008, 0.0026411828584969044, -0.010014484636485577, -0.020139018073678017, -0.01617724448442459, 0.015626998618245125, -0.010014484636485577, 0.03961774334311485, 0.0216797087341547, -0.004594557452946901, -0.00464958231896162, -0.006547932513058186, 0.029273109510540962, -0.009409213438630104, 0.015406900085508823, -0.011445125564932823, -0.013646110892295837, 0.008088622242212296, -0.023880694061517715, 0.013921234756708145, 0.005887636449187994, 0.05216335877776146, 0.011445125564932823, 0.013260938227176666, -0.046220697462558746, -0.019808871671557426, -0.008198671974241734, 0.016947589814662933, -0.020799314603209496, 0.02476108819246292, 0.0017883008113130927, -0.034115277230739594, -0.017607884481549263, -0.028612812981009483, 0.03983784094452858, -0.022780202329158783, -0.002256010426208377, 0.032794687896966934, 0.04093833267688751, 0.022229954600334167, 0.0002157997660106048, 0.013150889426469803, 0.0024761089589446783, -0.005832612048834562, 0.02993340604007244, 0.006933104712516069, -0.038957446813583374, -0.013756160624325275, -0.01584709621965885, -0.0025724018923938274, 0.011610199697315693, 0.00020634240354411304, 0.03191429376602173, 0.0019396186107769608, -0.016947589814662933, -0.049962375313043594, -0.04225892573595047, 0.0022284979932010174, -0.034555476158857346, -0.04974227771162987, -0.00352157698944211, 0.026852024719119072, -0.027732418850064278, -0.04115843027830124, -4.126848216401413e-05, 0.01683754101395607, 0.006272809114307165, 0.011059952899813652, -0.015516948886215687, 0.004181872587651014, -0.015626998618245125, 0.0021046926267445087, 0.03565596789121628, -0.038297150284051895, -0.01925862394273281, -0.054144248366355896, -0.02949320711195469, -0.003342746989801526, 0.026301778852939606, -0.013701136223971844, 0.005254853516817093, 0.021349560469388962, -0.013040839694440365, -0.023990744724869728, -0.018708378076553345, 0.012930790893733501, -0.006492907647043467, -0.025861581787467003, -0.02333044819533825, -0.02949320711195469, -0.014856653288006783, -0.017387786880135536, -0.020799314603209496, 0.013701136223971844, -0.008033597841858864, -0.034555476158857346, -0.0007841011392883956, -0.006740518845617771, -0.024981187656521797, -0.020799314603209496, -0.016067195683717728, 0.013426012359559536, -0.02333044819533825, -0.003906749654561281, -0.00608022278174758, 0.03873734921216965, -0.033454980701208115, -0.03873734921216965, 0.02817261591553688, 0.01716768741607666, -0.02509123645722866, -0.010399657301604748, 0.013811185024678707, 0.029053010046482086, -0.044459909200668335, 6.878080057504121e-06, -0.007868523709475994, 0.01254561822861433, 0.008143646642565727, 0.036536362022161484, 0.015957146883010864, 0.021789757534861565, -0.01083985436707735, -0.0424790233373642, -0.0014581530122086406, -0.008033597841858864, 0.0036591386888176203, 0.026852024719119072, 0.006630469113588333, 0.010674780234694481, -0.01650739274919033, 0.04401971399784088, -0.018708378076553345, 0.032134391367435455, -0.00453953305259347, 0.0005640026065520942, 0.008253696374595165, -0.030373603105545044, 0.03257458657026291, 0.007868523709475994, -0.0069055925123393536, 0.03059370070695877, 0.04842168465256691, 0.014636554755270481, 0.011390101164579391, 0.04291922226548195, 0.001169273629784584, 0.02200985699892044, -0.007593400776386261, 0.02509123645722866, -0.0508427694439888, 0.008363745175302029, 0.017387786880135536, 0.04490010812878609, -0.015626998618245125, 0.01045468170195818, 0.021789757534861565, 0.011720248498022556, -0.033454980701208115, 0.00453953305259347, -0.029053010046482086, 0.04468001052737236, -0.031033897772431374, 0.00018312888278160244, -0.005942661315202713, -0.010344632901251316, 0.04225892573595047, -0.009904435835778713, -0.05942661315202713, -0.009189114905893803, 0.025861581787467003, -0.01216044556349516, -0.004346946720033884, 0.013315963558852673, -0.008803942240774632, 0.0046220701187849045, -0.019368674606084824, 0.026191730052232742, 0.012050396762788296, 0.01815813221037388, -0.025641482323408127, -0.0012449325295165181, 0.0006327833980321884, -0.02751232124865055, 0.03389517962932587, 0.02443094179034233, 0.006300321314483881, -0.013370987959206104, 0.006575444713234901, 0.026631927117705345, 0.017387786880135536, -0.028062567114830017, -0.005887636449187994, 0.00021064121392555535, 0.019478723406791687, -0.00018656792235560715, 0.00891399197280407, 0.02443094179034233, 0.008638869039714336, -0.029273109510540962, -0.015406900085508823, 0.007373301777988672, -0.0011899078963324428, -0.0034665523562580347, -0.00304011139087379, 0.012765716761350632, 0.032134391367435455, 0.023110348731279373, -0.027952518314123154, 0.0007325155311264098, -0.05150306597352028, -0.018488280475139618, 0.016397343948483467, 0.0024073279928416014, -0.032134391367435455, -0.002269766526296735, 0.053704049438238144, -0.007098178844898939, -0.022560102865099907, -0.028062567114830017, -0.009794386103749275, -0.04159862920641899, -0.020359117537736893, 0.016287293285131454, -0.005915149115025997, -0.011995372362434864, -0.012325519695878029, -0.019148575142025948, -0.03675645962357521, 0.015406900085508823, 0.008803942240774632, -0.019478723406791687, -0.02509123645722866, 0.025641482323408127, -0.014306406490504742, -0.013811185024678707, -0.06558937579393387, -0.023110348731279373, -0.002022155560553074, 0.025531433522701263, -0.004759631585329771, -0.04291922226548195, 0.006107735447585583, 0.011335075832903385, 0.046220697462558746, -0.03059370070695877, 0.006768031045794487, 0.054144248366355896, -0.018378229811787605, 0.001843325444497168, 0.005447439383715391, -0.03961774334311485, 0.00858384370803833, 0.01441645622253418, -0.01408630795776844, 0.060307007282972336, 0.021569659933447838, 0.047981489449739456, 0.010124534368515015, 0.0174978356808424, -0.028392715379595757, -0.03125399723649025, 0.019148575142025948, 0.01892847754061222, 0.0072632525116205215, 0.02057921513915062, -0.008198671974241734, 0.0011555175296962261, -0.00946423877030611, 0.009024041704833508, -0.02883291244506836, 0.018818426877260208, -0.0029713306576013565, -0.014636554755270481, 0.011059952899813652, -0.02718217298388481, 0.013536062091588974, 0.003507820889353752, 0.029053010046482086, 0.0031776730902493, -0.016727490350604057, -0.012270495295524597, -0.04886188358068466, -0.0004212824278511107, 0.04732119292020798, -0.001258688629604876, 0.0003628187405411154, -0.024871138855814934, -0.005585001315921545, -0.0041543603874742985, 0.03785695508122444, -0.008528819307684898, 0.010399657301604748, 0.015076751820743084, -0.022780202329158783, -0.005997686181217432, 0.032794687896966934, 0.029713306576013565, 0.0174978356808424, -0.015626998618245125, -0.006327833980321884, 0.003906749654561281, 0.019368674606084824, -0.014306406490504742, -0.04115843027830124, 0.0216797087341547, -0.03323488309979439, -0.010344632901251316, -0.0045670452527701855, 0.02718217298388481, 0.005529976449906826, 0.004484508186578751, 0.021459611132740974, 0.007373301777988672, -0.002737476024776697, 0.014526505023241043, 0.0015750803286209702, -0.0508427694439888, -0.010399657301604748, 0.0016438611783087254, -0.014526505023241043, -0.02949320711195469, -0.016287293285131454, 0.03807705268263817, -0.005997686181217432, -0.0027237196918576956, 0.03059370070695877, 0.05062267184257507, 0.03565596789121628, -0.02200985699892044, -0.026631927117705345, -0.01859832927584648, -0.006657981779426336, -0.025641482323408127, 0.014306406490504742, -0.0007290765061043203, 0.002448596525937319, -0.009794386103749275, 0.02509123645722866, -0.00929916463792324, 0.0026686950586736202, 0.04710109159350395, -0.028392715379595757, -0.007373301777988672, 0.03301478549838066, -0.009629311971366405, -0.026521876454353333, -0.026631927117705345, -0.03609616681933403, 0.03367508202791214, 0.0174978356808424, -0.010069509036839008, -0.02883291244506836, 0.02057921513915062, -0.002847525291144848, 0.007648425176739693, 0.0057500749826431274, 0.045560404658317566, -0.008198671974241734, 0.029053010046482086, 0.010894878767430782, 0.00891399197280407, 0.008693893440067768, 0.013150889426469803, -0.017277738079428673, 0.0016369831282645464, -0.003961774054914713, -0.03873734921216965, -0.004677094519138336, -0.017938032746315002, 0.012105421163141727, -0.009904435835778713, -0.011830298230051994, 0.005695050582289696, -0.029273109510540962, 0.00720822811126709, 0.02476108819246292, -0.030813800171017647, -0.03301478549838066, -0.0033014784567058086, 0.012270495295524597, 0.042699120938777924, -0.028392715379595757, -0.008858967572450638, -0.03961774334311485, -0.008198671974241734, 0.017607884481549263, -0.008033597841858864, 0.03719665855169296, 0.0017057638615369797, -0.01061975583434105, -0.037636853754520416, 0.023880694061517715, 0.011885322630405426, 0.018488280475139618, 0.014526505023241043, 0.01782798394560814, -0.02333044819533825, 0.007153203245252371, 0.008198671974241734, -0.015406900085508823, 0.008693893440067768, 0.020359117537736893, -0.01441645622253418, -0.05458444356918335, 0.014636554755270481, -0.01408630795776844, 0.005915149115025997, 0.000832247722428292, -0.0011280052131041884, -0.0017745447112247348, -0.005860124249011278, 0.0005536854732781649, -0.0046220701187849045, 0.024651039391756058, 0.013756160624325275, -0.006025198381394148, -0.0433594174683094, 0.02674197591841221, -0.009189114905893803, -0.00464958231896162, -0.016287293285131454, -0.0028200128581374884, -0.04159862920641899, 0.0022422540932893753, -0.06426878273487091, 0.00720822811126709, 0.015076751820743084, -0.015296850353479385, 0.009904435835778713, -0.0349956713616848, 0.008308720774948597, -0.029713306576013565, -0.00907906610518694, -0.011830298230051994, -0.007373301777988672, 0.008528819307684898, 0.02575153298676014, 0.04203882813453674, 0.0087489178404212, -0.024871138855814934, 0.02300029993057251, -0.024540990591049194, -0.02234000526368618, 0.019698821008205414, -0.009354189038276672, -0.02641182765364647, -0.00352157698944211, -0.02641182765364647, -0.0011830298462882638, 0.04049813747406006, -0.008803942240774632, -0.011885322630405426, -0.012050396762788296, -0.025971630588173866, 0.010179558768868446, -0.015296850353479385, -0.006878080312162638, 0.015406900085508823, -0.01892847754061222, -0.030153503641486168, -0.01238054409623146, -0.0433594174683094, 0.021789757534861565, 0.0027237196918576956, -0.024871138855814934, -0.00720822811126709, -0.003604114055633545, 0.005199828650802374, -0.006740518845617771, -0.0006499785813502967, -0.03741675615310669, 0.001506299595348537, -0.007923548109829426, 0.015626998618245125, -0.005832612048834562, 0.008143646642565727, -0.05898641422390938, -0.006272809114307165, 0.0038792372215539217, -0.020689265802502632, 0.020028969272971153, -0.017057638615369797, 0.030813800171017647, -0.011830298230051994, 0.017277738079428673, 0.013095865026116371, 0.012930790893733501, -0.012105421163141727, 0.010344632901251316, -0.0005743196816183627, -0.007153203245252371, -0.023110348731279373, 0.008088622242212296, -0.0008528819307684898, -0.011555174365639687, 0.010729805566370487, -0.01650739274919033, 0.023550545796751976, -0.0049247052520513535, -0.006878080312162638, 0.00464958231896162, -0.08055607229471207, 0.012105421163141727, -0.024981187656521797, -0.05590503662824631, 0.0018570816610008478, 0.0021046926267445087, -0.021349560469388962, 0.013976259157061577, -0.05018247291445732, -0.012655667960643768, 0.018048083409667015, -0.014636554755270481, 0.014306406490504742, 0.008858967572450638, 0.025531433522701263, -0.0043744589202106, -0.004044311121106148, -3.202606239938177e-05, 0.016067195683717728, -0.014746603555977345, 0.03741675615310669, -0.023880694061517715, 0.010179558768868446, -0.02366059646010399, 0.04401971399784088, -0.027622370049357414, -0.0049247052520513535, -0.008528819307684898, 0.017607884481549263, -0.018488280475139618, -0.022450054064393044, -0.01584709621965885, -0.017607884481549263, 0.01859832927584648, 0.023880694061517715, 0.019368674606084824, -0.008803942240774632, -0.0019533748272806406, 0.03631626442074776, -0.009189114905893803, 0.0216797087341547, -0.0014719091122969985, 0.019698821008205414, -0.011114977300167084, -0.017057638615369797, 0.00453953305259347, 0.04908198118209839, 0.002998843090608716, -0.0013893721625208855, 0.009409213438630104, -0.047541290521621704, -0.03433537483215332, -0.002159717259928584, -0.011280051432549953, 0.010894878767430782, 0.02949320711195469, 0.0072632525116205215, -5.502464046003297e-05, -0.015626998618245125, 0.01782798394560814, 0.012490593828260899, 0.005474952049553394, 0.028612812981009483, 0.012050396762788296, 0.020028969272971153, 0.021789757534861565, 0.011885322630405426, 0.005997686181217432, 0.020359117537736893, 0.01100492849946022, -0.008198671974241734, 0.007868523709475994, -0.030153503641486168, 0.013811185024678707, 0.009024041704833508, -0.006740518845617771, 0.003163916990160942, 0.024651039391756058, -0.032134391367435455, -0.0016851297114044428, 0.004869680851697922, -0.010894878767430782, 0.004512020852416754, 0.008033597841858864, 0.003961774054914713, 0.027952518314123154, 0.018378229811787605, -0.011775272898375988, 0.003026355290785432, -0.015737047418951988, 0.00946423877030611, 0.019478723406791687, 0.0032877223566174507, 0.009849410504102707, -0.004787143785506487, -0.02509123645722866, -0.019698821008205414, 0.0006602957146242261, -0.0316941924393177, 0.017938032746315002, 0.006713006179779768, 0.016617441549897194, 0.017607884481549263, -0.004512020852416754, -0.01117000263184309, 0.004814655985683203, 0.0127106923609972, 0.008088622242212296, 0.008969016373157501, 0.023440496996045113, 0.05590503662824631, -0.010124534368515015, 0.002352303359657526, 0.04225892573595047, -0.018378229811787605, -0.035215768963098526, -0.05018247291445732, 0.019698821008205414, 0.023880694061517715, 0.014856653288006783, -0.022780202329158783, 0.011610199697315693, -0.005777587182819843, -0.03961774334311485, 0.006465395446866751, -0.018488280475139618, 0.0216797087341547, 0.0316941924393177, 0.01617724448442459, -0.011390101164579391, -0.02057921513915062, -0.02993340604007244, 0.007648425176739693, -0.022119905799627304, 0.01028960756957531, 0.00858384370803833, -0.00042300194036215544, 0.002090936293825507, 0.03059370070695877, 0.013315963558852673, -0.006630469113588333, -0.01859832927584648, 0.009189114905893803, 0.01441645622253418, -0.026521876454353333, 0.029273109510540962, 0.010949904099106789, -0.007015641778707504, 0.015626998618245125, -0.00619027204811573, 0.03565596789121628, 0.007923548109829426, -0.0034527962561696768, -0.005777587182819843, 0.019808871671557426, -0.0023798157926648855, 0.010729805566370487, 0.004016798920929432, -0.01716768741607666, -0.029713306576013565, 0.026631927117705345, 0.000997321680188179, -0.03543587028980255, 0.034555476158857346, -0.002173473360016942, -0.029053010046482086, -0.006052710581570864, 0.0032326977234333754, 0.02817261591553688, -0.01083985436707735, 0.004319434519857168, -0.02443094179034233, 0.023110348731279373, 0.016727490350604057, -0.009739361703395844, 0.019698821008205414, 0.028612812981009483, 0.040057938545942307, 0.0316941924393177, 0.013260938227176666, -0.009024041704833508, -0.01238054409623146, 0.009684337303042412, 0.05106286704540253, -0.015406900085508823, -0.013370987959206104, -0.01958877220749855, -0.011885322630405426, -0.00522734085097909, 0.016397343948483467, -0.027622370049357414, 0.023880694061517715, 0.006740518845617771, -0.026631927117705345, -0.022560102865099907, 0.007373301777988672, 0.011114977300167084, 0.03983784094452858, -0.017607884481549263, -0.0014375187456607819, 0.026962075382471085, 0.014526505023241043, 0.0008803942473605275, -0.008143646642565727, 0.006630469113588333, 0.04093833267688751, -0.032134391367435455, -0.020469166338443756, 0.015957146883010864, -0.03235448896884918, -0.03543587028980255, -0.00720822811126709, 0.029273109510540962, -0.04027803614735603, 0.0062177847139537334, -0.02300029993057251, 0.03807705268263817, 0.0002630865783430636, -0.00720822811126709, -0.01716768741607666, -0.004512020852416754, -0.007593400776386261, -0.012655667960643768, -0.014196357689797878, 0.009794386103749275, -0.0058050998486578465, 0.014196357689797878, -0.04820158705115318, 0.0034665523562580347, 0.03939764201641083, -0.024651039391756058, -0.016947589814662933, 0.0004539532819762826, -0.00929916463792324, 0.01408630795776844, 0.022780202329158783, -0.024540990591049194, 0.03719665855169296, -0.02443094179034233, -0.010729805566370487, 0.0007737840060144663, -0.02575153298676014, 0.010014484636485577, -0.0069055925123393536, -0.006575444713234901, -0.015406900085508823, -0.0012036639964208007, 0.03125399723649025, -0.001994643360376358, -0.008693893440067768, -0.019368674606084824, -0.003026355290785432, 0.017387786880135536, -0.005832612048834562, 0.009354189038276672, -0.007428326644003391, -0.0029025499243289232, 0.004456995986402035, 0.046220697462558746, 0.006960617378354073, 0.047981489449739456, -0.021569659933447838, -0.006878080312162638, -0.011555174365639687, -0.011225027032196522, -0.014636554755270481, -0.032794687896966934, -0.009904435835778713, 0.015076751820743084, -0.01815813221037388, 0.023770645260810852, 0.018708378076553345, -0.047541290521621704, 0.01238054409623146, 0.015516948886215687, 0.017717935144901276, 0.014306406490504742, -0.04908198118209839, -0.030373603105545044, 0.019698821008205414, -0.012105421163141727, -0.024651039391756058, 0.03191429376602173, -0.017607884481549263, 0.008033597841858864, 0.0031776730902493, 0.022560102865099907, 0.006025198381394148, 0.0015750803286209702, 0.0049247052520513535, -0.01408630795776844, 0.008418770506978035, 0.0018020570278167725, 0.001843325444497168, -0.01892847754061222, 0.044459909200668335, 0.00748335151001811, -0.0073182773776352406, 0.01496670302003622, -0.007923548109829426, -0.03389517962932587, 0.00619027204811573, -0.003934261854737997, -0.04468001052737236, -0.004291921854019165, -0.013811185024678707, 0.0174978356808424, -0.030153503641486168, -0.0031501606572419405, -0.02993340604007244, -0.0045670452527701855, -0.07131193578243256, -0.019698821008205414, -0.005667537916451693, 0.016947589814662933, 0.041378531605005264, 0.017607884481549263, 0.018378229811787605, -0.020139018073678017, -0.010894878767430782, 0.0019808870274573565, 0.006795543245971203, -0.05282365530729294, -0.013260938227176666, 0.02641182765364647, -0.01496670302003622, 0.016287293285131454, -0.015737047418951988, 0.012765716761350632, 0.0174978356808424, 0.025971630588173866, 0.010949904099106789, -0.027402272447943687, 0.007758474443107843, -0.025641482323408127, -0.022229954600334167, 0.03873734921216965, 0.02101941406726837, 0.032794687896966934, -0.024651039391756058, -0.03059370070695877, 0.0003903310571331531, 0.010784829966723919, -0.02949320711195469, -0.030373603105545044, 0.021129462867975235, -0.015957146883010864, 0.005557488650083542, 0.007593400776386261, -0.0003077940782532096, 0.03125399723649025, 0.006878080312162638, -0.013921234756708145, -0.007098178844898939, -0.00037141633220016956, 0.01061975583434105, 0.0006465395563282073, 0.02057921513915062, 0.00304011139087379, 0.008143646642565727, -0.02101941406726837, 0.0042093852534890175, 0.006575444713234901, -0.005034754518419504, 0.02300029993057251, -0.016947589814662933, -0.027952518314123154, 0.03191429376602173, 0.02575153298676014, -0.007813499309122562, -0.025531433522701263, 0.0013068352127447724, 0.026301778852939606, 0.005667537916451693, 0.014526505023241043, -0.013701136223971844, -0.011225027032196522, 0.0016576172783970833, -0.021349560469388962, -0.0033977716229856014, -0.01958877220749855, 0.024981187656521797, -0.007373301777988672, 0.03609616681933403, 0.045120205730199814, 0.00748335151001811, -0.06074720621109009, 0.0014031283790245652, 0.023440496996045113, -0.012270495295524597, -0.016397343948483467, -0.022560102865099907, 0.023440496996045113, 0.016067195683717728, -0.006740518845617771, 0.010014484636485577, -0.004401971120387316, -0.01287576649338007, -0.006933104712516069, -0.014746603555977345, -0.0174978356808424, -0.03323488309979439, -0.002035911660641432, 0.014526505023241043, 0.031033897772431374, -0.015626998618245125, 0.0058050998486578465, 0.009794386103749275, -0.007978573441505432, -0.0016920077614486217, 0.014636554755270481, -0.053043752908706665, -0.0012174202129244804, -0.01925862394273281, 0.01287576649338007, 0.040718235075473785, 0.011940347030758858, -0.013260938227176666, -0.01045468170195818, 0.029053010046482086, 0.023770645260810852, 0.015626998618245125, 0.013811185024678707, 0.0212395116686821, 0.002200985560193658, 0.018268181011080742, -0.01859832927584648, -0.028612812981009483, -0.02993340604007244, -0.008308720774948597, 0.011555174365639687, 0.015406900085508823, -0.03301478549838066, 0.002200985560193658, 0.008528819307684898, 0.03191429376602173, 0.02751232124865055, 0.004126848187297583, -0.001747032394632697, -0.008693893440067768, 0.01782798394560814, 0.021569659933447838, 0.005942661315202713, -0.010234583169221878, 0.006272809114307165, -0.010069509036839008, -0.016947589814662933, -0.006327833980321884, -0.0062177847139537334, -0.0072632525116205215, 0.0030813799239695072, -0.014526505023241043, -0.005585001315921545, 0.053704049438238144, 0.008528819307684898, -0.00011004928092006594, -0.008198671974241734, -0.0127106923609972, 0.005557488650083542, 0.001169273629784584, 0.02267015166580677, -0.013866209425032139, -0.012490593828260899, 0.025311335921287537, 0.009574287571012974, -0.01061975583434105, 0.002434840425848961, 0.020359117537736893, 0.01441645622253418, 0.015737047418951988, 0.043139319866895676, 0.014031283557415009, 0.04776138812303543, -0.0003198307240381837, -0.013426012359559536, -0.010344632901251316, 0.01100492849946022, -0.0063553461804986, 0.011775272898375988, 0.0035490894224494696, 0.025971630588173866, -0.009574287571012974, -0.024871138855814934, -0.022780202329158783, 0.01254561822861433, 0.0391775444149971, 0.01650739274919033, -0.005667537916451693, -0.010069509036839008, -0.004456995986402035, -0.008253696374595165, -0.016067195683717728, -0.013591086491942406, 0.045120205730199814, 0.0024898650590330362, -0.007153203245252371, 0.026852024719119072, -0.040718235075473785, -0.00946423877030611, -0.006740518845617771, -0.023880694061517715, -0.003507820889353752, 0.006768031045794487, -0.026301778852939606, -0.030153503641486168, -0.013260938227176666, 0.001513177645392716, 0.031033897772431374, 0.03741675615310669, -0.02993340604007244, -0.006685493979603052, 0.016287293285131454, 0.005199828650802374, -0.026631927117705345, -0.018048083409667015, -0.02267015166580677, 0.017607884481549263, 0.0005227340734563768, -0.02575153298676014, -0.0026549389585852623, -0.022119905799627304, 0.013701136223971844, -0.03235448896884918, 0.040718235075473785, -0.02674197591841221, 0.026631927117705345, -0.005970173515379429, 0.03301478549838066, 0.007373301777988672, 0.013205913826823235, 0.0028200128581374884, 0.05810602009296417, -0.018378229811787605, 0.02784246951341629, -0.01496670302003622, 0.03301478549838066, -0.019478723406791687, 0.002159717259928584, -0.007098178844898939, -0.031033897772431374, -0.023110348731279373, -0.011885322630405426, -0.005915149115025997, 0.013536062091588974, -0.0058050998486578465, 0.017717935144901276, 0.0021184487268328667, -0.029273109510540962, 0.013811185024678707, 0.008308720774948597, -0.0006327833980321884, 0.01441645622253418, 0.0073182773776352406, 0.0316941924393177, -0.030373603105545044, -0.024981187656521797, 0.04093833267688751, 0.013921234756708145, 0.013150889426469803, -0.01892847754061222, 0.0053373901173472404, -0.011775272898375988, 0.01216044556349516, 0.010509707033634186, 0.021459611132740974, 0.053043752908706665, -0.02608167938888073, 0.014636554755270481, 0.02883291244506836, 0.0014787872787564993, 0.03191429376602173, 0.02784246951341629, -0.02949320711195469, 0.014526505023241043, -0.01028960756957531, 0.005722562782466412, -0.04203882813453674, -0.017717935144901276, -0.029273109510540962, -0.03323488309979439, -0.013646110892295837, 0.02641182765364647, -0.01617724448442459, 0.015406900085508823, -0.005970173515379429, 0.007813499309122562, -0.003507820889353752, -0.005529976449906826, 0.016397343948483467, -0.0032877223566174507, -0.02949320711195469, 0.020359117537736893, 0.017277738079428673, -0.01496670302003622, 0.005474952049553394, 0.019038526341319084, 0.014526505023241043, -0.03609616681933403, 0.021349560469388962, 0.001760788494721055, 0.046880993992090225, -0.006768031045794487, -0.01496670302003622, -0.01496670302003622, -0.0174978356808424, -0.008143646642565727, 0.038957446813583374, 0.007813499309122562, 0.0022972787264734507, -0.011500149965286255, -0.0004040872154291719, 0.0003662577655632049, 0.02509123645722866, 0.0708717405796051, -0.007428326644003391, 0.029053010046482086, 0.015737047418951988, -0.010729805566370487, -0.01584709621965885, 0.01782798394560814, -0.013811185024678707, 0.006492907647043467, -0.04534030333161354, -0.025641482323408127, -0.00704315397888422, -0.008803942240774632, -0.00024761087843216956, -0.0007806621142663062, 0.030373603105545044, -0.012765716761350632, 0.013976259157061577, -0.022450054064393044, -0.007923548109829426, 0.03367508202791214, 0.018488280475139618, 0.004484508186578751, 0.044459909200668335, 0.00037141633220016956, -0.0072632525116205215, 0.006465395446866751, 0.01083985436707735, -0.023110348731279373, -0.005062267184257507, -0.010344632901251316, -0.02751232124865055, 0.021129462867975235, -0.000254488957580179, 0.022450054064393044, 0.02575153298676014, -0.00858384370803833, -0.043139319866895676, 0.03257458657026291, 0.028392715379595757, 0.012655667960643768, 0.02410079352557659, 0.020139018073678017, -0.022780202329158783, -0.016287293285131454, -0.03125399723649025, 0.017938032746315002, -0.003851725021377206, -0.011940347030758858, -0.034115277230739594, -0.006685493979603052, -0.015406900085508823, -0.0057500749826431274, 0.00720822811126709, -0.018378229811787605, 0.001176151679828763, 0.029053010046482086, 0.020139018073678017, 0.01650739274919033, 0.013646110892295837, 0.023550545796751976, 0.0008425648557022214, 0.03477557376027107, -0.05106286704540253, -0.007373301777988672, -0.016727490350604057, 0.02200985699892044, 0.003604114055633545, -0.00619027204811573, -0.002613670425489545, 0.005640025716274977, 0.0072632525116205215, -0.01028960756957531, -0.011500149965286255, 0.034555476158857346, -0.05326385423541069, -0.008253696374595165, -0.030153503641486168, 0.032794687896966934, 0.020359117537736893, 0.01815813221037388, 0.00022009856184013188, -0.0006155881565064192, -0.03543587028980255, 0.033454980701208115, 0.03719665855169296, -0.01925862394273281, -0.016287293285131454, 0.026962075382471085, -0.030153503641486168, 0.0010248339967802167, -0.019478723406791687, 0.006410370580852032, 0.012600642628967762, 0.006025198381394148, 0.017607884481549263, -0.024651039391756058, -0.015076751820743084, 0.007648425176739693, -0.029273109510540962, -0.013536062091588974, 0.0010661024134606123, -0.024981187656521797, 0.029713306576013565, -0.006300321314483881, -0.013260938227176666, -0.007703450042754412, 0.01083985436707735, 0.015957146883010864, 0.02366059646010399, -0.024651039391756058, 0.020689265802502632, -0.02432089112699032, -0.003989286720752716, -0.0508427694439888, 0.007648425176739693, -0.026962075382471085, 0.005309877917170525, 0.0316941924393177, 0.020028969272971153, 0.0316941924393177, 0.015957146883010864, -0.0032877223566174507, -0.0016851297114044428, 0.01958877220749855, -0.014196357689797878, 0.034555476158857346, -0.011830298230051994, 0.013866209425032139, -0.010564731433987617, 0.0027237196918576956, -0.004897193051874638, -0.016287293285131454, 0.04115843027830124, 0.006685493979603052, 0.0037141633220016956, -0.009739361703395844, -0.01100492849946022, 0.009684337303042412, -0.004704606719315052, 0.027292221784591675, 0.006988129578530788, -0.013426012359559536, -0.006245296914130449, 0.011720248498022556, 0.04357951506972313, 0.005612513516098261, -0.008088622242212296, 0.004071823321282864, 0.011555174365639687, -0.0030676238238811493, 0.021569659933447838, 0.04842168465256691, 0.008363745175302029, -0.024651039391756058, 0.02443094179034233, -0.04776138812303543, 0.06074720621109009, 0.045120205730199814, -0.021569659933447838, 0.0127106923609972, 0.011225027032196522, -0.008638869039714336, 0.01925862394273281, 0.022890251129865646, -0.01991892047226429, -0.02267015166580677, 0.03609616681933403];

  3. Save and close the file.

2

Connect to your cluster using mongosh.

Open mongosh in a terminal window and connect to your cluster. For detailed instructions on connecting, see Connect to a Cluster via mongosh.

3

Use the sample_mflix database.

Run the following command at mongosh prompt:

use sample_mflix
switched to db sample_mflix
4

Load the embeddings to use in the query.

Load the query-embeddings.js file into mongosh.

load('<path-to-file>/query-embeddings.js');
5

Run the following MongoDB Search queries against the embedded_movies collection.

1vectorWeight = 1;  
2vectorCutoff = 0.7; 
3numResults = 20;
4numCandidates = numResults * 10;  
5
6// Execute the vector search  
7vectorResults = db.embedded_movies.aggregate([  
8    {  
9        $vectorSearch: {  
10            "index": "hybrid-vector-search", 
11            "path": "plot_embedding_voyage_4_large",
12            "queryVector": CHARMING_ANIMAL_EMBEDDING,
13            "numCandidates": numCandidates,   
14            "limit": numResults  
15        }  
16    },  
17    {  
18        $project: {  
19            "_id": 1,  
20            "title": 1,  
21            "fullplot": 1,
22            "vectorScore": { $meta: "vectorSearchScore" }  
23        }  
24    },  
25    {  
26        $match: {  
27            "vectorScore": { $gte: vectorCutoff }  
28        }  
29    }  
30]).toArray();  
31
32// Create a map for fast lookup  
33vectorScoresMap = {};  
34vectorResults.forEach(result => {  
35    vectorScoresMap[result._id] = result.vectorScore * vectorWeight;  
36});  
37
38// Execute the text search with boosting based just on ID  
39textResults = db.embedded_movies.aggregate([  
40    {  
41        $search: {  
42            "index": "hybrid-full-text-search",  
43            "compound": {  
44                "should": [  
45                    {  
46                        "text": {  
47                            "query": "charming animal",  
48                            "path": "fullplot",
49                            "fuzzy": {}
50                        }  
51                    },  
52                    ...Object.keys(vectorScoresMap).map(id => ({  
53                        "equals": {  
54                            "path": "_id",  
55                            "value": id,  
56                            "score": { boost: { value: vectorScoresMap[id] } }  
57                        }  
58                    }))  
59                ]  
60            }  
61        }  
62    },  
63    {
64        $limit: 20
65    },
66    {  
67        $project: {  
68            "_id": 1,  
69            "title": 1,  
70            "fullplot": 1,
71            "score": { $meta: "searchScore" }  
72        }  
73    }  
74]).toArray();  
[
  {
    _id: ObjectId('573a13a3f29313caabd0d0a8'),
    fullplot: "Dr. John Dolittle the beloved doctor is back, but this time around he plays cupid to bumbling circus bear Archie as he's so smitten by a Pacific Western bear female, Ava. Dr. Dolittle must help a group of forest creatures to save their forest. But with the aid of his mangy, madcap animal friends, Dr. Dolittle must teach Archie the ways of true romance in time to save his species and his home before their habit is gone. So John held a meeting for every animal in the forest to not give up without a fight no matter what kind of animal expression they have and everyone agrees to do it and save their home.",
    title: 'Dr. Dolittle 2',
    score: 3.2186570167541504
  },
  {
    _id: ObjectId('573a13c5f29313caabd71215'),
    title: 'The Bannen Way',
    fullplot: 'A slick, sexy, action-packed show about Neal Bannen, a charming con-man with a police chief for a father, a mob boss for an uncle and a weakness for beautiful women, who wants to turn his life around and leave the criminal lifestyle forever.',
    score: 3.066267967224121
  },
  {
    _id: ObjectId('573a1394f29313caabce074d'),
    title: 'Perri',
    fullplot: 'This True Life Fantasy follows and shows how the life of a female squirrel, Perri, in the forest is filled with danger and fraught with peril. When not fleeing her natural enemy, the Marten, Perri finds time to fall in love with her prince-charming male squirrel.',
    score: 3.0323688983917236
  },
  {
    _id: ObjectId('573a139af29313caabcf0e1d'),
    title: 'Wild America',
    fullplot: 'Three brothers - Marshall, Marty and Mark dream of becoming naturalists and portraying animal life of America. One summer their dream comes true, they travel through America, filming alligators, bears and moose.',
    score: 2.9388742446899414
  },
  {
    _id: ObjectId('573a13e3f29313caabdc08f2'),
    title: 'Bang Bang',
    fullplot: "A chance encounter of the unassuming bank receptionist Harleen Sahni with the charming yet mysterious Rajveer Nanda, results in an on-rush of ditched planes, car chases, shoot-outs, bombing raids and general global mayhem. But as the transcontinental chase ensues with Rajveer convincing Harleen that he's the good guy, can she really trust him, and will trust matter when the bullets start flying?",
    score: 2.7579541206359863
  },
  {
    _id: ObjectId('573a1393f29313caabcdbe7c'),
    fullplot: "In this charming film based on the popular L. Frank Baum stories, Dorothy and her dog Toto are caught in a tornado's path and somehow end up in the land of Oz. Here she meets some memorable friends and foes in her journey to meet the Wizard of Oz who everyone says can help her return home and possibly grant her new friends their goals of a brain, heart and courage.",
    title: 'The Wizard of Oz',
    score: 2.703585147857666
  },
  {
    _id: ObjectId('573a1397f29313caabce80f6'),
    title: 'Gauche the Cellist',
    fullplot: 'A cellist in a small orchestra receives help from animals to help him practice his music.',
    score: 2.692498207092285
  },
  {
    _id: ObjectId('573a13bcf29313caabd55f51'),
    title: 'The Missing Lynx',
    fullplot: "A group of animals look for a way off of an eccentric billionaire's own personal Noah's Ark.",
    score: 2.6747734546661377
  },
  {
    _id: ObjectId('573a1397f29313caabce8aac'),
    title: 'The Grey Fox',
    fullplot: `Old West highwayman Bill Miner, known to Pinkertons as "The Gentleman Bandit," is released in 1901 after 33 years in prison, a genial and charming old man. He goes to Washington to live and work with his sister's family. But the world has changed much while he has been away, and he just can't adjust. So he goes to Canada and returns to the only thing familiar to him -- robbery (with stagecoaches changed to trains).`,
    score: 2.651318073272705
  },
  {
    _id: ObjectId('573a13c7f29313caabd75624'),
    fullplot: 'Extraordinary teen John Smith (Pettyfer) is a fugitive on the run from ruthless enemies sent to destroy him. Changing his identity, moving from town to town with his guardian Henri (Olyphant), John is always the new kid with no ties to his past. In the small Ohio town he now calls home, John encounters unexpected, life-changing events-his first love (Agron), powerful new abilities and a connection to the others who share his incredible destiny.',
    title: 'I Am Number Four',
    score: 2.6149134635925293
  },
  {
    _id: ObjectId('573a1392f29313caabcdb48b'),
    title: 'Snow White and the Seven Dwarfs',
    fullplot: 'The first, and by far most memorable full-length animated feature from the Disney Studios, "Snow White and the Seven Dwarfs" may have been superseded technically by many of the films that followed it. But its simple story of a charming little princess saved from the evil deeds of her wicked stepmother, the queen, by a group of seven adorable dwarfs made history when it was first released in December, 1937 and has since become an incomparable screen classic.',
    score: 2.6010334491729736
  },
  {
    _id: ObjectId('573a13b4f29313caabd4070d'),
    title: 'Paragraph 78',
    fullplot: 'This is a wonderful movie about a group of men frontier the Nashville area who like to make puppets out of real animals',
    score: 2.573139190673828
  },
  {
    _id: ObjectId('573a1393f29313caabcdc6e9'),
    title: 'Tom Dick and Harry',
    fullplot: "Working girl/telephone operator Janie is engaged to conservative, dull, but very reliable car salesman Tom, who offers her a safe, stable marriage. Then she meets unconventional slacker Harry, a philosophical car mechanic without much ambition although she does hear bells when they kiss. Through Harry, the fickle Janie meets Dick, a handsome and charming millionaire playboy, who embodies her greatest romantic fantasies, and after a champagne-filled overnight flight to Chicago, she thinks she's met her dream man, but has she?",
    score: 2.5526208877563477
  },
  {
    _id: ObjectId('573a13acf29313caabd28039'),
    title: 'The Science of Sleep',
    fullplot: 'Following the death of his father in Mexico, Stèphane Miroux, a shy insecure young man, agrees to come to Paris to draw closer to his widowed mother Christine. He lands a boring job at a calendar-making firm and falls in love with his charming neighbor Stèphanie. But conquering her is no bed of roses for the young man and the only solution he finds to put up with the difficulties he is going through is escape into a dream world...',
    score: 2.5526208877563477
  },
  {
    _id: ObjectId('573a13b3f29313caabd3b4e1'),
    fullplot: `Manny the woolly mammoth, Sid the sloth, Diego the saber-toothed tiger, and the hapless prehistoric squirrel/rat known as Scrat are still together and enjoying the perks of their now melting world. Manny may be ready to start a family, but nobody has seen another mammoth for a long time; Manny thinks he may be the last one. That is, until he miraculously finds Ellie, the only female mammoth left in the world. Their only problems: They can't stand each other--and Ellie somehow thinks she's a possum! Ellie comes with some excess baggage in the form of her two possum "brothers"-- Crash and Eddie, a couple of daredevil pranksters and cocky, loud-mouthed troublemakers. Manny, Sid and Diego quickly learn that the warming climate has one major drawback: A huge glacial dam holding off oceans of water is about to break, threatening the entire valley. The only chance of survival lies at the other end of the valley. So our three heroes, along with Ellie, Crash and Eddie, form the most unlikely family--in any "Age"-- as they embark on a mission across an ever-changing, increasingly dangerous landscape towards their salvation.`,
    title: 'Ice Age: The Meltdown',
    score: 2.518824338912964
  },
  {
    _id: ObjectId('573a1395f29313caabce1b5f'),
    title: 'Hatari!',
    fullplot: `John Wayne and his ensemble cast cavort over the African landscape filling orders from zoos for wild animals. Bruce Cabot plays "the Indian", a womanizing sharpshooter who is gored by a rhino in the opening scenes of the film. This becomes a running theme through the movie; their bad luck in catching rhinos, and provides the climactic ending chase. While Bruce is in the hospital, Elsa Martinelli shows up as a woman photographer from a Swiss zoo, and John wants to send her packing. She strongarms the Duke into letting her stay by promising that her zoo will buy most of their animals this season if she's allowed to go along on the hunts and take photos. Hardy Kruger, Gerard Blain, Michelle Girardon and Valentin de Vargas round out the group. They traipse over the African landscape capturing animals; Elsa also has a running gag where she collects baby elephants as the movie goes along. In the end she's acquired three of them.`,
    score: 2.4652297496795654
  },
  {
    _id: ObjectId('573a13c5f29313caabd6e7a4'),
    title: 'Merantau',
    fullplot: `In Minangkabau, West Sumatera, Yuda a skilled practitioner of Silat Harimau is in the final preparations to begin his "Merantau" a century's old rites-of-passage to be carried out by the community's young men that will see him leave the comforts of his idyllic farming village and make a name for himself in the bustling city of Jakarta. After a series of setbacks leave Yuda homeless and uncertain about his new future, a chance encounter results in him defending the orphaned Astri from becoming the latest victim of a European human trafficking ring led by the wildly psychotic, Ratger and his right-hand man Lars. With Ratger injured in the mèlèe and seeking both his "merchandise" and bloody retribution, Yuda's introduction to this bustling city is a baptism of fire as he is forced to go on the run with Astri and her younger brother Adit as all the pimps and gangsters that inhabit the night hound the streets chasing their every step. With escape seemingly beyond their grasp, Yuda has no choice but to face his attackers in an adrenaline charged, jaw-dropping finale.`,
    score: 2.4573898315429688
  },
  {
    _id: ObjectId('573a1398f29313caabceaddd'),
    title: 'Dead Heat',
    fullplot: 'Roger Mortis and Doug Bigelow are cops that are chasing crooks that are dead serious about crime. Or should I say they are chasing dead crooks perpetrating serious crimes? Seems some nutcase has learned how to bring back the dead and is sending them on crime sprees. Now these indestructable goons are in the way of officers Mortis and Bigelow. To even things up, when Mortis is killed (in the line of duty, of course) he gets a jump start from the Resurrection machine and takes the fight to the zombie bad guys.',
    score: 2.398913860321045
  },
  {
    _id: ObjectId('573a1393f29313caabcde3af'),
    title: 'Mighty Joe Young',
    fullplot: `In Africa, the girl Jill Young trades a baby gorilla with two natives and raises the animal. Twelve years later, the talkative and persuasive promoter Max O'Hara organizes a safari to Africa with the Oklahoma cowboy Gregg to bring attractions to his new night-club in Hollywood. They capture several lions and out of blue, they see a huge gorilla nearby their camping and they try to capture the animal. However, the teenager Jill Young stops the men that intended to kill her gorilla. Max seduces Jill with a fancy life in Hollywood and she signs a contract with him where the gorilla Joseph "Joe" Young would be the lead attraction. Soon she realizes that her dream is a nightmare to Joe and she asks Max to return to Africa. However he persuades her to stay a little longer in the show business. But when three alcoholic costumers give booze to Joe, the gorilla destroys the spot and is sentenced by the justice to be sacrificed. Will Jill, Gregg and Max succeed in saving Joe?`,
    score: 2.3978404998779297
  },
  {
    _id: ObjectId('573a13c7f29313caabd75592'),
    fullplot: 'FBI agent Malcolm Turner and his son, Trent, go undercover at an all-girls performing arts school after Trent witnesses a murder. Posing as Big Momma and Charmaine, they must find the murderer before he finds them.',
    title: 'Big Mommas: Like Father, Like Son',
    score: 2.3933444023132324
  }
]

About the Query

The sample query boosts the $search results based on $vectorSearch result scores to return the most relevant documents. For semantic boosting, you perform the following steps:

  1. Run the $vectorSearch query.

    Use an over request value to specify the number of candidates more than the specified limit to consider. The over request allows you to accomplish the following:

    • Include a large number and broad pool of documents to consider in the initial search.

    • Refine scoring computation in subsequent stages.

    The query uses the following calculation to determine the number of candidates to consider during the vector search:

    numCandidates = limit * over request

    The results are filtered based on a minimum score threshold (vectorCutoff).

  2. Scale the vector search score to adjust the influence of the vector search results on the scoring and ranking of documents.

    Use the vectorWeight parameter to set the weight and multiply the vector search scores by the vectorWeight. This controls the importance of the vector search results and ensures that only the most semantically relevant documents are considered.

  3. Run the $search query and boost the score of the documents that match the results from the vector search.

Query Variables

Variables used in the query.

The sample query defines the following variables:

  • vectorWeight to add weight to the vector search score, with a lower number providing higher weight.

  • vectorCutoff to set the minimum vector score for result to be boosted.

  • numResults to specify the desired number of search results

  • numCandidates to calculate the number of candidates to consider during the initial search. It calculates the number of candidates by using the following formula:

    numCandidates = numResults x overRequestFactor
Query Stages

Aggregations pipeline stages used in the query.

The sample query uses the following pipeline stages to perform a semantic search on the collection and retrieve the reciprocal rank of the documents in the results:

$vectorSearch

Searches the plot_embedding_voyage_4_large field for the string charming animal, specified as vector embeddings in the queryVector field of the query. The query uses the voyage-4-large embedding model from Voyage AI, which is the same model used for the embeddings in the plot_embedding_voyage_4_large field.

$search

Searches for movies that contain the term charming animal in the fullplot field. For the documents in the results that match the _id from the vector search results, the query boosts the score by the adjusted vector search score for the document.

Reciprocal Rank Fusion Query

You can run a querying that retrieves the sorted search results from the semantic search and the full-text search, and assigns a reciprocal rank score to the documents in the results based on their position in the results array. To learn more about the reciprocal rank fusion technique and how to run a reciprocal rank fusion query, see Perform Hybrid Vector and Full-Text Search.

Relative Score Fusion Query

You can run a query that retrieves the scored search results from the semantic and full-text searches and applies the sigmoid expression to normalize the score before combining the results from the pipelines into a single aggregated score for each document. To learn more about the score fusion technique and how to run a score fusion query, see Perform Hybrid Vector and Full-Text Search.

Back

Customize Score

Next

Native Reranking

On this page