Docs Menu
Docs Home
Docs Home
Docs Home

Generate Embeddings Automatically Using MongoDB Vector Search

You can configure MongoDB Vector Search to automatically generate and manage vector embeddings for the text data in your cluster. You can create a one-click AI semantic search index in your M10 or higher Atlas cluster and use Voyage AI embedding models, simplifying indexing, updating, and querying with vectors.

When you enable automated embeddings, MongoDB Vector Search automatically generates embeddings using the specified embedding model at index-time for the specified text field in your Atlas collection and at query-time for your text string in your query against the field indexed for automated embeddings.

Considerations

Important

You can use MongoDB Vector Search automated embeddings on any M10 or higher cluster on any cloud provider. However, the service that handles the inference process for generating vector embeddings runs on Google Cloud. This means that your data is sent to Google Cloud for embedding generation and retrieval, regardless of your cluster's cloud provider. We provide enterprise grade security and therefore, your data is only stored in your cluster.

The embedding models run on a shared, multi-tenant inference platform. Therefore, during the preview period, you must use datasets with less than 100k document and run queries only for the evaluation of the feature and not for load testing. Contact your account team if you have a use case with higher limits.

Although there are no hard rate-limits for your workload, there are global limits. If your queries return a rate limit error (error 409), perform a backoff and retry in your application code. This allows your application to gracefully handle rate limits and ensures continued functionality.

Prerequisites

To enable vector search using automated embeddings, you must have the following:

  • M10 or higher cluster

  • A collection with a text field that you want to index for automated embeddings.

  • One of the following clients:

    • Atlas UI for creating indexes

    • mongosh for creating indexes and running queries

    • Node Driver 6.6.0 or higher for creating indexes and running queries

    • Python Driver 4.7 or higher for creating indexes and running queries

    Use the following drop-down menu to set the client for the procedures on this page.

MongoDB Vector Search Index for Automated Embeddings

The following sections describe the MongoDB Vector Search index syntax and fields for enabling automatic generation of embeddings for text fields and walk you through the steps for configuring your index for automated embeddings.

Required Access

You need the Project Data Access Admin or higher role to create and manage MongoDB Vector Search indexes.

Index Syntax

The following is the syntax for enabling automatic generation of embeddings:

1{
2  "fields": [
3    {
4      "type": "text",
5      "path": "<field-name>",
6      "model": "voyage-3-large | voyage-3.5 | voyage-3.5-lite"
7    }
8  ]
9}

Index Fields

The following fields are required in the index definition:

Field

Type

Description

type

string

The type of the field. For automated embeddings, this must be text.

path

string

The name of the field in the collection that you want to index for automated embeddings.

model

string

The Voyage AI embedding model to use for generating the embeddings for the index. You can specify one of the following models:

  • voyage-3-large - Highest-quality retrieval across languages and domains.

  • voyage-3.5 - Balanced model for multilingual use and general-purpose retrieval accuracy.

  • voyage-3.5-lite - Lightweight, faster model optimized for latency and lower cost.

After creating the index, if you change the embedding model subsequently, MongoDB Vector Search generates new embeddings for the dataset. While MongoDB Vector Search generates the embeddings, you can continue to query by using the old embeddings. When the old embeddings are replaced with embeddings from the new embedding model, MongoDB Vector Search removes the old embeddings.

Considerations

The index fields for automated embeddings are mutually exclusive with the following vector type index fields:

  • numDimensions

  • similarity

  • quantization

If your collection already has embeddings, you must use the vector type fields to index the embeddings. To learn more about indexing fields with embeddings, see How to Index Fields for Vector Search.

You can create an index with both the text and vector types if you want to index a text field for automatically generating embeddings and also index a field with your own embeddings. MongoDB Vector Search will automatically generate embeddings for queries against only the field indexed as the text type. You must specify embeddings in the query for searching the field indexed as the vector type.

You can also index fields to pre-filter your data by using the MongoDB Vector Search filter type. Pre-filtering reduces the number of documents against which to run similarity comparisons, which can decrease query latency and increase the accuracy of search results. To learn more about pre-filtering your data, see About the filter Type.

Create an Index for Automated Embeddings

The following procedure walks through the steps for enabling automated embeddings in your MongoDB Vector Search index. If you loaded the sample_mflix.movies dataset, the example in the procedure demonstrates how to enable automated embeddings for the fullplot field in the collection.

1

In Atlas, go to the Clusters page for your project.

  1. If it's not already displayed, select the organization that contains your desired project from the Organizations menu in the navigation bar.

  2. If it's not already displayed, select your desired project from the Projects menu in the navigation bar.

  3. In the sidebar, click Clusters under the Database heading.

The Clusters page displays.

2

In Atlas, go to the Search & Vector Search page for your cluster.

You can go the MongoDB Search page from the Search & Vector Search option, or the Data Explorer.

  1. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar.

  2. If it's not already displayed, select your project from the Projects menu in the navigation bar.

  3. In the sidebar, click Search & Vector Search under the Database heading.

    If you have no clusters, click Create cluster to create one. To learn more, see Create a Cluster.

  4. If your project has multiple clusters, select the cluster you want to use from the Select cluster dropdown, then click Go to Atlas Search.

    The Search & Vector Search page displays.

  1. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar.

  2. If it's not already displayed, select your project from the Projects menu in the navigation bar.

  3. In the sidebar, click Data Explorer under the Database heading.

  4. Expand the database and select the collection.

  5. Click the Indexes tab for the collection.

    The Atlas Search page displays.

3

Start your index configuration.

Make the following selections on the page and then click Next.

Search Type

Select the Vector Search index type.

Index Name and Data Source

Specify the following information:

  • Index Name: <INDEX-NAME>

  • Database and Collection:

    • <DATABASE-NAME>

    • <COLLECTION-NAME>

Configuration Method

Select JSON Editor.

Example

For example, enter movies_automated_embeddings for the index name and select sample_mflix and movies from the Database and Collection dropdown.

4

Define your index.

In the JSON editor, enter the following index definition:

1{
2  "fields": [
3    {
4      "type": "text",
5      "path": "<FIELD-NAME>",
6      "model": "voyage-3-large | voyage-3.5 | voyage-3.5-lite"
7    }
8  ]
9}

Example

For example, to create an index that enables automated embeddings by using the voyage-3-large model for the fullplot field in the collection, enter the following:

1{
2  "fields": [
3    {
4      "type": "text",
5      "path": "fullplot",
6      "model": "voyage-3-large"
7    }
8  ]
9}
5

Create your index.

  1. Click Next.

  2. Review your index configuration and click Create Vector Search Index.

Note

The index creation can take some time to complete. When the index is being created, the Status column in the Atlas UI displays Pending. When the index creation completes, the Status column in the Atlas UI displays Ready.

1

Connect to the Atlas cluster using mongosh.

To learn more, see Connect to a Cluster via mongosh.

2

Switch to the database that contains the collection for which you want to create the index.

To switch, run the following command:

use <DATABASE-NAME>

Here, <DATABASE-NAME> is the name of the database you want to use.

Example

For example, to switch to the sample_mflix database, which contains the movies collection, run the following command:

use sample_mflix
3

Run the db.collection.createSearchIndex() method.

The db.collection.createSearchIndex() method has the following syntax:

1db.<COLLECTION-NAME>.createSearchIndex(
2  "<INDEX-NAME>",
3  "vectorSearch", 
4  {
5    "fields": [
6      {
7        "type": "text",
8        "path": "<FIELD-NAME>",
9        "model": "voyage-3-large | voyage-3.5 | voyage-3.5-lite"
10      },
11      {
12        "type": "filter",
13        "path": "<FIELD-NAME>"
14      },
15      ...
16    ]
17  }
18);

Example

For example, to create an index that enables automated embeddings by using the voyage-3-large model for the fullplot field in the collection, run the following command in your terminal:

1db.movies.createSearchIndex(
2  "movies_automated_embeddings",
3  "vectorSearch",
4  {
5    "fields": [
6      {
7        "type": "text",
8        "path": "fullplot",
9        "model": "voyage-3-large"
10      }
11    ]
12  }
13);
1

Initialize your Node.js project.

Run the following commands in your terminal to create a new directory named auto-embeddings-project and initialize your project:

mkdir auto-embeddings-project
cd auto-embeddings-project
npm init -y
2

Create the .js file.

touch <file-name>.js

Example

For example, create a file named create_index.js.

touch create_index.js
3

Define the index in the .js file.

1const { MongoClient } = require("mongodb");
2
3// connect to your Atlas deployment
4const uri =  "<CONNECTION-STRING>";
5const client = new MongoClient(uri);
6
7async function run() {
8    try {
9      const database = client.db("<DATABASE-NAME>");
10      const collection = database.collection("<COLLECTION-NAME>");
11
12      // define your MongoDB Vector Search index
13      const index = {
14        "name": "<INDEX-NAME>",
15        "type": "vectorSearch",
16        "definition": {
17          "fields": [
18            {
19              "type": "vector",
20              "path": "<FIELD-NAME>",
21              "model": "voyage-3-large | voyage-3.5 | voyage-3.5-lite"
22            },
23          ]
24        }
25      }
26      // run the helper method
27      await collection.createSearchIndex("<INDEX-NAME>", index);
28    } finally {
29      await client.close();
30    }
31}
32run().catch(console.dir);

Example

For example, to create an index that enables automated embeddings by using the voyage-3-large model for the fullplot field in the sample_mflix.movies namespace, copy and paste the following in the create_index.js file:

1const { MongoClient } = require("mongodb");
2
3// connect to your Atlas deployment
4const uri =  "<CONNECTION-STRING>";
5const client = new MongoClient(uri);
6
7async function run() {
8  try {
9    const database = client.db("sample_mflix");
10    const collection = database.collection("movies");
11
12    // define your MongoDB Vector Search index
13    const index = {
14      "name": "movies_automated_embeddings",
15      "type": "vectorSearch",
16      "definition": {
17      "fields": [
18        {
19          "type": "text",
20          "path": "fullplot",
21          "model": "voyage-3-large"
22        }
23      ]
24    }
25  }
26  // run the helper method
27  await collection.createSearchIndex(index);
28  } finally {
29    await client.close();
30  }
31}
32run().catch(console.dir);
4

Replace the following values and save the file.

<CONNECTION-STRING>

The connection string for your Atlas cluster.

<DATABASE-NAME>

The name of the database that contains the collection for which you want to create the index.

<COLLECTION-NAME>

The name of the collection for which you want to create the index.

<FIELD-NAME>

The name of the field in the collection that you want to index for automated embeddings.

<INDEX-NAME>

The name of the index that you want to create.

5

Run the following command to create the index.

node <file-name>.js

Example

For example, to create the index defined in the create_index.js file, run the following command in your terminal:

node create_index.js
1

Create a .py file.

touch <file-name>.py

Example

For example, create a file named create_index.py.

touch create_index.py
2

Define the index in the .py file.

1from pymongo import MongoClient
2from pymongo.operations import SearchIndexModel
3
4client = pymongo.MongoClient("<CONNECTION-STRING>")
5db = client["<DATABASE-NAME>"]
6collection = db["<COLLECTION-NAME>"]
7
8search_index_model = SearchIndexModel(
9  definition={
10    "fields": [
11      {
12        "type": "text",
13        "path": "<FIELD-NAME>",
14        "model": "voyage-3-large | voyage-3.5 | voyage-3.5-lite" 
15      }
16    ]
17  },
18  name="<INDEX-NAME>",
19  type="vectorSearch",
20)
21result = collection.create_search_index(model=search_index_model)
22print(result)

Example

For example, to create an index that enables automated embeddings by using the voyage-3-large model for the fullplot field in the sample_mflix.movies namespace, copy and paste the following in the create_index.py file:

1from pymongo import MongoClient
2from pymongo.operations import SearchIndexModel
3
4client = MongoClient("<CONNECTION-STRING>")
5db = client["sample_mflix"]
6collection = db["movies"]
7
8search_index_model = SearchIndexModel(
9  definition={
10    "fields": [
11      {
12        "type": "text",
13        "path": "fullplot",
14        "model": "voyage-3-large"
15      }
16    ]
17  },
18    name="movies_automated_embeddings",
19    type="vectorSearch",
20)
21result = collection.create_search_index(model=search_index_model)
22print(result)
3

Replace the following values and save the file.

<CONNECTION-STRING>

The connection string for your Atlas cluster.

<database-name>

The name of the database that contains the collection for which you want to create the index.

<collection-name>

The name of the collection for which you want to create the index.

<field-name>

The name of the field in the collection that you want to index for automated embeddings.

<index-name>

The name of the index that you want to create.

4

Run the following command to create the index.

python <file-name>.py

Example

For example, to create the index defined in the create_index.py file, run the following command in your terminal:

python create_index.py

MongoDB Vector Search Query with Automated Embeddings

After you create an index with automated embeddings, you can run text queries against the indexed field. MongoDB Vector Search automatically generates embeddings for the text string in your query using the same embedding model specified in the index. It uses the embeddings to search the index for documents that are semantically similar to the specified query text.

The following sections describe the $vectorSearch pipeline syntax and fields for automatically generating embeddings for your query text against the field indexed for automated embeddings and demonstrate how to run semantic search queries against the fields indexed for automated embeddings.

Query Syntax

The following syntax demonstrates how to run a query against a field indexed for automated embeddings:

1{
2  "$vectorSearch": {
3    "index": "<index-name>",
4    "limit": <number-of-results>,
5    "numCandidates": <number-of-candidates>,
6    "path": "<field-to-search>",
7    "query": "<query-string>"
8  }
9}

Query Fields

The following fields are required for a MongoDB Vector Search query using automated embeddings:

Field
Type
Necessity
Description

exact

boolean

Conditional

This field is required if numCandidates is omitted. Mutually exclusive with numCandidates.

Flag that specifies whether to run ENN or ANN search. Value can be one of the following:

  • false - to run ANN search

  • true - to run ENN search

If omitted, defaults to false.

index

string

Required

Name of the MongoDB Vector Search index to use.

MongoDB Vector Search doesn't return results if you misspell the index name or if the specified index doesn't already exist on the cluster.

limit

number

Required

Number (of type int only) of documents to return in the results. This value can't exceed the value of numCandidates if you specify numCandidates.

numCandidates

number

Conditional

This field is required if exact is false or omitted. Mutually exclusive with exact.

Number of nearest neighbors to use during the search. Value must be less than or equal to (<=) 10000. You can't specify a number less than the number of documents to return (limit).

path

string

Required

Indexed vector type field to search.

query

string

Required

Text for which to automatically generate embeddings and perform the semantic search.

Considerations

You can run an ANN or ENN query against the indexed field. To learn more, see ANN Search and ENN Search.

You can't specify vector embeddings in your query against fields indexed for automated embeddings. Instead, you must run a natural language query against the field. When you run a natural language query against the field indexed for automated embeddings, MongoDB Vector Search automatically generates the embeddings for the query text using the same embedding model as the indexed field. It then uses the generated embeddings to perform a semantic search against the indexed field.

You can optionally specify filter fields in your query to pre-filter the documents against which MongoDB Vector Search performs the semantic search. To learn more, see MongoDB Vector Search Pre-Filter.

You can also optionally retrieve the score of the documents in the results. To learn more, see MongoDB Vector Search Score.

Run a Query Using Automated Embeddings

You can't run MongoDB Vector Search queries in the Atlas UI. Use the mongosh or a supported driver to run queries.

1

Connect to the Atlas cluster using mongosh.

To learn more, see Connect to a Cluster via mongosh.

2

Switch to the database that contains the collection for which you want to run the query.

use <database-name>

Example

For example, to switch to the sample_mflix database, run the following command in your terminal:

use sample_mflix
3

Run the db.collection.aggregate() method with the $vectorSearch stage.

The db.collection.aggregate() method has the following syntax:

1db.<COLLECTION-NAME>.aggregate([
2  {
3    "$vectorSearch": {
4      "index": "<INDEX-NAME>", 
5      "path": "<FIELD-NAME>", 
6      "query": "<QUERY-TEXT>", 
7      "numCandidates": <NUMBER-OF-CANDIDATES-TO-CONSIDER>, 
8      "limit": <NUMBER-OF-DOCUMENTS-TO-RETURN>
9    }
10  }
11])

Example

For example, to run a query against the fullplot field in the movies collection for a semantic search for movies semantically similar to young heroes caught in epic struggles between light and darkness, copy, paste, and run the following query.

1db.movies.aggregate([
2  {
3    "$vectorSearch": {
4      "index": "movies_automated_embeddings", 
5      "path": "fullplot", 
6      "query": "young heroes caught in epic struggles between light and darkness", 
7      "numCandidates": 1000, 
8      "limit": 10
9    }
10  },
11  {
12    "$project": {
13      "_id": 0, 
14      "title": 1, 
15      "fullplot": 1, 
16      "score": {"$meta": "vectorSearchScore"}
17    }
18  }
19])
1[
2   {
3       title: 'Day Watch',
4       fullplot: 'Anton belongs to the Forces of the Light as well as his powerful girlfriend and apprentice, but his son is a powerful teenager from the Darkness and Anton protects him. When the balance between Light and Darkness is affected by the death of some evil vampires, Anton is framed and accused of the murders, and he chases an ancient chalk that has the power of changing the destiny of its owner.',
5       score: 0.5449697971343994
6   },
7   {
8       title: 'Dungeons & Dragons',
9       fullplot: "The Empire of Izmer has long been a divided land. The Mages - an elite group of magic-users - rule whilst the lowly commoners are powerless. Izmer's young Empress, Savina, wants equality and prosperity for all, but the evil Mage Profion is plotting to depose her and establish his own rule. The Empress possesses a scepter which controls Izmer's Golden Dragons. To challenge her rule, Profion must have the scepter, and tricks the Council of Mages into believing Savina is unfit to hold it. Knowing that Profion will bring death and destruction to Izmer, Savina must find the legendary Rod of Savrille, a mythical rod that has the power to control Red Dragons, a species even mightier than the Gold. Enter two thieves, Ridley and Snails, who unwittingly become instrumental in Savina's search for the Rod. Joined by a feisty Dwarf named Elwood, and helped by the Empress's expert tracker, the Elf Norda, the young heroes go in search of the Rod of Savrille. From the deadly maze of the Thieves Guild at Antius to an Elven Village, secret grotto and abandoned castles, Ridley and his band must outwit Profion's chief henchman Damodar at every turn while, back in Izmer, Profion prepares to do battle with the Empress. All depends on the Rod, but the outcome of the race to reach it first is far from certain, and Izmar's very survival hangs in the balance.",
10       score: 0.5414832830429077
11   },
12   {
13       title: 'Brave Story',
14       fullplot: 'A young boy attempts to change his destiny by entering a magic gateway to another world; but on his quest to find the Tower of Fortune and be granted any wish, he must conjure up all his bravery to battle demons, his friends, and ultimately himself.',
15       score: 0.5404887795448303
16   },
17   {
18       title: 'Justin and the Knights of Valour',
19       fullplot: 'Justin lives in a kingdom where bureaucrats rule and knights have been ousted. His dream is to be become one of the Knights of Valour, like his grandfather was, but his father Reginald, the chief counsel to the Queen, wants his son to follow in his footsteps and become a lawyer. After an inspiring visit to his beloved Grandmother and bidding farewell to his supposed lady-love Lara, Justin leaves home and embarks on a quest to become a knight. Along the way he meets the beautiful, feisty Talia, a quirky wizard called Melquiades, and the handsome Sir Clorex and is mentored by three monks; Blucher, Legantir and Braulio, who teach and test him in the ancient ways of the Knights of Valour. Whilst an unlikely candidate for knighthood, Justin must rise to the challenge quickly when banished former knight Sir Heraclio and his army, lead by Sota, return and threaten to destroy the Kingdom.',
20       score: 0.5374966859817505
21   },
22   {
23       title: 'Forest Warrior',
24       fullplot: 'John McKenna is a spiritual being who is able to transform into bear, wolf or eagle. He lives in the forests of Tanglewood and has dedicated his life to protect them. One day a gang of evil lumberjacks led by Travis Thorne arrive Tanglewood to chop the forest down. McKenna cannot let this happen, and together with his new friends - Lords of the Tanglewood, a band of children who love to play in the forest - he battles against Thorne and his evil gang.',
25       score: 0.5331881642341614
26   },
27   {
28       title: 'Forest Warrior',
29       fullplot: 'John McKenna is a spiritual being who is able to transform into bear, wolf or eagle. He lives in the forests of Tanglewood and has dedicated his life to protect them. One day a gang of evil lumberjacks led by Travis Thorne arrive Tanglewood to chop the forest down. McKenna cannot let this happen, and together with his new friends - Lords of the Tanglewood, a band of children who love to play in the forest - he battles against Thorne and his evil gang.',
30       score: 0.5331881642341614
31   },
32   {
33       title: 'Catatan (Harian) si Boy',
34       fullplot: "A circle of friends risking their Friendship, Trust, Love and Hope in search of a legend. A young and privileged teenager with a golden heart, beset with challenges and tribulations we face today with the goal to open many young people's mind with inspirations and hopes that drive them in achieving their dreams. To get out of their comfort zone and finish what they started.",
35       score: 0.5322973728179932
36   },
37   {
38       title: 'Bionicle: Mask of Light',
39       fullplot: "In a land of living machines, two young ones are chosen to seek the legendary Mask of Light to reveal the savior of all the lands from the dark forces of the Makuta. During the course of their adventure, they will call on the heroes of their people, the great Toa. These Toa, masters of nature's forces such as Fire, Wind, Earth & Water, try to protect the chosen ones as they seek their destiny.",
40       score: 0.5315042734146118
41   },
42   {
43       title: 'Fear No Evil',
44       fullplot: 'High school student turns out to be personification of Lucifer. Two arch angels in human form (as women) take him on.',
45       score: 0.5295513868331909
46   },
47   {
48       title: 'Tales of Vesperia: The First Strike',
49       fullplot: 'In a mythical kingdom, the mighty Imperial Knights harness a magical substance known as Aer to power their weapons and protect humanity from the monsters of the forest. But something strange is afoot. The Aer is somehow changing, causing the wilderness to waste away and stirring the woodland beasts to attack with greater frequency. As danger creeps steadily closer to civilization, two young recruits - Flynn, the rigid son of a fallen hero, and the rebellious and brash Yuri - must ride with their fellow Imperial Knights to distant ruins in hopes of uncovering the truth behind the transforming Aer. Some will not survive the thrilling journey. Some will be betrayed. If Flynn and Yuri cannot overcome their differences and learn to fight together, all will be lost for the people of the realm.',
50       score: 0.5276793241500854
51   }
52]
1

Create a .js file.

Example

For example, create a file named automated-embeddings-query.js.

touch automated-embeddings-query.js
2

Define the query in the file.

1const { MongoClient } = require("mongodb");
2
3// connect to your Atlas cluster
4const uri = "<CONNECTION-STRING>";
5
6const client = new MongoClient(uri);
7
8async function run() {
9    try {
10        await client.connect();
11
12        // set namespace
13        const database = client.db("<DATABASE-NAME>");
14        const coll = database.collection("<COLLECTION-NAME>");
15
16        // define pipeline
17        const agg = [
18            {
19              '$vectorSearch': {
20                'index': '<INDEX-NAME>',
21                'path': '<FIELD-NAME>',
22                'query': "<QUERY-TEXT>",
23                'numCandidates': <NUMBER-OF-CANDIDATES-TO-CONSIDER>,
24                'limit': <NUMBER-OF-DOCUMENTS-TO-RETURN>
25              }
26            }
27          ];
28        // run pipeline
29        const result = coll.aggregate(agg);
30
31        // print results
32        await result.forEach((doc) => console.dir(JSON.stringify(doc)));
33    } finally {
34        await client.close();
35    }
36}
37run().catch(console.dir);

Example

For example, in the automated-embeddings-query.js file, copy paste the following code to define a query against the fullplot field in the movies collection for a semantic search for movies semantically similar to young heroes caught in epic struggles between light and darkness.

1const { MongoClient } = require("mongodb");
2
3// connect to your Atlas cluster
4const uri = "<CONNECTION-STRING>";
5
6const client = new MongoClient(uri);
7
8async function run() {
9    try {
10        await client.connect();
11
12        // set namespace
13        const database = client.db("sample_mflix");
14        const coll = database.collection("movies");
15
16        // define pipeline
17        const agg = [
18            {
19              '$vectorSearch': {
20                'index': 'movies_automated_embeddings',
21                'path': 'fullplot',
22                'query': "young heroes caught in epic struggles between light and darkness",
23                'numCandidates': 1000,
24                'limit': 10
25              }
26            }, {
27              '$project': {
28                '_id': 0,
29                'fullplot': 1,
30                'title': 1,
31                'score': {
32                  '$meta': 'vectorSearchScore'
33                }
34              }
35            }
36          ];
37        // run pipeline
38        const result = coll.aggregate(agg);
39
40        // print results
41        await result.forEach((doc) => console.dir(JSON.stringify(doc)));
42    } finally {
43        await client.close();
44    }
45}
46run().catch(console.dir);
3

Replace the following in the query.

<CONNECTION-STRING>

You cluster connection string.

<DATABASE-NAME>

Name of the database that contains the collection.

<COLLECTION-NAME>

Name of the collection that contains the indexed field.

<INDEX-NAME>

Name of the index.

<FIELD-NAME>

Name of the indexed field.

<QUERY-TEXT>

Text string for which to generate embeddings and use in the semantic search.

<NUMBER-OF-CANDIDATES-TO-CONSIDER>

Number of nearest neighbors to consider.

<NUMBER-OF-DOCUMENTS-TO-RETURN>

Number of documents to return in the results.

4

Run the query.

node <FILE-NAME>.js

Here <FILE-NAME> is the name of the .js file you created.

Example

For example, to run the example query in the automated-embeddings-query.js file, run the following command:

node automated-embeddings-query.js
'{"title":"Day Watch","fullplot":"Anton belongs to the Forces of the Light as well as his powerful girlfriend and apprentice, but his son is a powerful teenager from the Darkness and Anton protects him. When the balance between Light and Darkness is affected by the death of some evil vampires, Anton is framed and accused of the murders, and he chases an ancient chalk that has the power of changing the destiny of its owner.","score":0.5449697971343994}'
`{"title":"Dungeons & Dragons","fullplot":"The Empire of Izmer has long been a divided land. The Mages - an elite group of magic-users - rule whilst the lowly commoners are powerless. Izmer's young Empress, Savina, wants equality and prosperity for all, but the evil Mage Profion is plotting to depose her and establish his own rule. The Empress possesses a scepter which controls Izmer's Golden Dragons. To challenge her rule, Profion must have the scepter, and tricks the Council of Mages into believing Savina is unfit to hold it. Knowing that Profion will bring death and destruction to Izmer, Savina must find the legendary Rod of Savrille, a mythical rod that has the power to control Red Dragons, a species even mightier than the Gold. Enter two thieves, Ridley and Snails, who unwittingly become instrumental in Savina's search for the Rod. Joined by a feisty Dwarf named Elwood, and helped by the Empress's expert tracker, the Elf Norda, the young heroes go in search of the Rod of Savrille. From the deadly maze of the Thieves Guild at Antius to an Elven Village, secret grotto and abandoned castles, Ridley and his band must outwit Profion's chief henchman Damodar at every turn while, back in Izmer, Profion prepares to do battle with the Empress. All depends on the Rod, but the outcome of the race to reach it first is far from certain, and Izmar's very survival hangs in the balance.","score":0.5414832830429077}`
'{"title":"Brave Story","fullplot":"A young boy attempts to change his destiny by entering a magic gateway to another world; but on his quest to find the Tower of Fortune and be granted any wish, he must conjure up all his bravery to battle demons, his friends, and ultimately himself.","score":0.5404887795448303}'
'{"title":"Justin and the Knights of Valour","fullplot":"Justin lives in a kingdom where bureaucrats rule and knights have been ousted. His dream is to be become one of the Knights of Valour, like his grandfather was, but his father Reginald, the chief counsel to the Queen, wants his son to follow in his footsteps and become a lawyer. After an inspiring visit to his beloved Grandmother and bidding farewell to his supposed lady-love Lara, Justin leaves home and embarks on a quest to become a knight. Along the way he meets the beautiful, feisty Talia, a quirky wizard called Melquiades, and the handsome Sir Clorex and is mentored by three monks; Blucher, Legantir and Braulio, who teach and test him in the ancient ways of the Knights of Valour. Whilst an unlikely candidate for knighthood, Justin must rise to the challenge quickly when banished former knight Sir Heraclio and his army, lead by Sota, return and threaten to destroy the Kingdom.","score":0.5374966859817505}'
'{"title":"Forest Warrior","fullplot":"John McKenna is a spiritual being who is able to transform into bear, wolf or eagle. He lives in the forests of Tanglewood and has dedicated his life to protect them. One day a gang of evil lumberjacks led by Travis Thorne arrive Tanglewood to chop the forest down. McKenna cannot let this happen, and together with his new friends - Lords of the Tanglewood, a band of children who love to play in the forest - he battles against Thorne and his evil gang.","score":0.5331881642341614}'
'{"title":"Forest Warrior","fullplot":"John McKenna is a spiritual being who is able to transform into bear, wolf or eagle. He lives in the forests of Tanglewood and has dedicated his life to protect them. One day a gang of evil lumberjacks led by Travis Thorne arrive Tanglewood to chop the forest down. McKenna cannot let this happen, and together with his new friends - Lords of the Tanglewood, a band of children who love to play in the forest - he battles against Thorne and his evil gang.","score":0.5331881642341614}'
`{"title":"Catatan (Harian) si Boy","fullplot":"A circle of friends risking their Friendship, Trust, Love and Hope in search of a legend. A young and privileged teenager with a golden heart, beset with challenges and tribulations we face today with the goal to open many young people's mind with inspirations and hopes that drive them in achieving their dreams. To get out of their comfort zone and finish what they started.","score":0.5322973728179932}`
`{"title":"Bionicle: Mask of Light","fullplot":"In a land of living machines, two young ones are chosen to seek the legendary Mask of Light to reveal the savior of all the lands from the dark forces of the Makuta. During the course of their adventure, they will call on the heroes of their people, the great Toa. These Toa, masters of nature's forces such as Fire, Wind, Earth & Water, try to protect the chosen ones as they seek their destiny.","score":0.5315042734146118}`
'{"title":"Fear No Evil","fullplot":"High school student turns out to be personification of Lucifer. Two arch angels in human form (as women) take him on.","score":0.5295513868331909}'
'{"title":"Tales of Vesperia: The First Strike","fullplot":"In a mythical kingdom, the mighty Imperial Knights harness a magical substance known as Aer to power their weapons and protect humanity from the monsters of the forest. But something strange is afoot. The Aer is somehow changing, causing the wilderness to waste away and stirring the woodland beasts to attack with greater frequency. As danger creeps steadily closer to civilization, two young recruits - Flynn, the rigid son of a fallen hero, and the rebellious and brash Yuri - must ride with their fellow Imperial Knights to distant ruins in hopes of uncovering the truth behind the transforming Aer. Some will not survive the thrilling journey. Some will be betrayed. If Flynn and Yuri cannot overcome their differences and learn to fight together, all will be lost for the people of the realm.","score":0.5276793241500854}'
1

Create a .py file.

Example

For example, create a file named automated-embeddings-query.js.

touch automated-embeddings-query.py
2

Define the query in the file.

1import pymongo
2
3# connect to your Atlas cluster
4client = pymongo.MongoClient("<CONNECTION-STRING>")
5
6# define pipeline
7pipeline = [
8  {
9    '$vectorSearch': {
10      'index': '<INDEX-NAME>', 
11      'path': '<FIELD-NAME>', 
12      'query': '<QUERY-TEXT>', 
13      'numCandidates': <NUMBER-OF-CANDIDATES-TO-CONSIDER>,
14      'limit': <NUMBER-OF-DOCUMENTS-TO-RETURN>
15    }
16  }
17]
18
19# run pipeline
20result = client["<DATABASE-NAME>"]["<COLLECTION-NAME>"].aggregate(pipeline)
21
22# print results
23for i in result:
24    print(i)
25

Example

For example, in the automated-embeddings-query.py file, copy paste the following code to define a query against the fullplot field in the movies collection for a semantic search for movies semantically similar to young heroes caught in epic struggles between light and darkness.

1import pymongo
2
3# connect to your Atlas cluster
4client = pymongo.MongoClient("<CONNECTION-STRING>")
5
6# define pipeline
7pipeline = [
8  {
9    '$vectorSearch': {
10      'index': 'movies_automated_embeddings',
11      'path': 'fullplot',
12      'query': "young heroes caught in epic struggles between light and darkness",
13      'numCandidates': 1000,
14      'limit': 10
15    },
16  },
17  {
18    '$project': {
19      '_id': 0,
20      'title': 1,
21      'fullplot': 1,
22      'score': {'$meta': 'vectorSearchScore'}
23    }
24  }
25]
26
27# run pipeline
28result = client["sample_mflix"]["movies"].aggregate(pipeline)
29
30# print results
31for i in result:
32    print(i)
33
3

Replace the following in the query.

<CONNECTION-STRING>

You cluster connection string.

<DATABASE-NAME>

Name of the database that contains the collection.

<COLLECTION-NAME>

Name of the collection that contains the indexed field.

<INDEX-NAME>

Name of the index.

<FIELD-NAME>

Name of the indexed field.

<QUERY-TEXT>

Text string for which to generate embeddings and use in the semantic search.

<NUMBER-OF-CANDIDATES-TO-CONSIDER>

Number of nearest neighbors to consider.

<NUMBER-OF-DOCUMENTS-TO-RETURN>

Number of documents to return in the results.

4

Run the query.

python <FILE-NAME>.py

Here <FILE-NAME> is the name of the .py file you created.

Example

For example, to run the example query in the automated-embeddings-query.py file, run the following command:

python automated-embeddings-query.js
{'title': 'Day Watch', 'fullplot': 'Anton belongs to the Forces of the Light as well as his powerful girlfriend and apprentice, but his son is a powerful teenager from the Darkness and Anton protects him. When the balance between Light and Darkness is affected by the death of some evil vampires, Anton is framed and accused of the murders, and he chases an ancient chalk that has the power of changing the destiny of its owner.', 'score': 0.5449697971343994}
{'title': 'Dungeons & Dragons', 'fullplot': "The Empire of Izmer has long been a divided land. The Mages - an elite group of magic-users - rule whilst the lowly commoners are powerless. Izmer's young Empress, Savina, wants equality and prosperity for all, but the evil Mage Profion is plotting to depose her and establish his own rule. The Empress possesses a scepter which controls Izmer's Golden Dragons. To challenge her rule, Profion must have the scepter, and tricks the Council of Mages into believing Savina is unfit to hold it. Knowing that Profion will bring death and destruction to Izmer, Savina must find the legendary Rod of Savrille, a mythical rod that has the power to control Red Dragons, a species even mightier than the Gold. Enter two thieves, Ridley and Snails, who unwittingly become instrumental in Savina's search for the Rod. Joined by a feisty Dwarf named Elwood, and helped by the Empress's expert tracker, the Elf Norda, the young heroes go in search of the Rod of Savrille. From the deadly maze of the Thieves Guild at Antius to an Elven Village, secret grotto and abandoned castles, Ridley and his band must outwit Profion's chief henchman Damodar at every turn while, back in Izmer, Profion prepares to do battle with the Empress. All depends on the Rod, but the outcome of the race to reach it first is far from certain, and Izmar's very survival hangs in the balance.", 'score': 0.5414832830429077}
{'title': 'Brave Story', 'fullplot': 'A young boy attempts to change his destiny by entering a magic gateway to another world; but on his quest to find the Tower of Fortune and be granted any wish, he must conjure up all his bravery to battle demons, his friends, and ultimately himself.', 'score': 0.5404887795448303}
{'title': 'Justin and the Knights of Valour', 'fullplot': 'Justin lives in a kingdom where bureaucrats rule and knights have been ousted. His dream is to be become one of the Knights of Valour, like his grandfather was, but his father Reginald, the chief counsel to the Queen, wants his son to follow in his footsteps and become a lawyer. After an inspiring visit to his beloved Grandmother and bidding farewell to his supposed lady-love Lara, Justin leaves home and embarks on a quest to become a knight. Along the way he meets the beautiful, feisty Talia, a quirky wizard called Melquiades, and the handsome Sir Clorex and is mentored by three monks; Blucher, Legantir and Braulio, who teach and test him in the ancient ways of the Knights of Valour. Whilst an unlikely candidate for knighthood, Justin must rise to the challenge quickly when banished former knight Sir Heraclio and his army, lead by Sota, return and threaten to destroy the Kingdom.', 'score': 0.5374966859817505}
{'title': 'Forest Warrior', 'fullplot': 'John McKenna is a spiritual being who is able to transform into bear, wolf or eagle. He lives in the forests of Tanglewood and has dedicated his life to protect them. One day a gang of evil lumberjacks led by Travis Thorne arrive Tanglewood to chop the forest down. McKenna cannot let this happen, and together with his new friends - Lords of the Tanglewood, a band of children who love to play in the forest - he battles against Thorne and his evil gang.', 'score': 0.5331881642341614}
{'title': 'Forest Warrior', 'fullplot': 'John McKenna is a spiritual being who is able to transform into bear, wolf or eagle. He lives in the forests of Tanglewood and has dedicated his life to protect them. One day a gang of evil lumberjacks led by Travis Thorne arrive Tanglewood to chop the forest down. McKenna cannot let this happen, and together with his new friends - Lords of the Tanglewood, a band of children who love to play in the forest - he battles against Thorne and his evil gang.', 'score': 0.5331881642341614}
{'title': 'Catatan (Harian) si Boy', 'fullplot': "A circle of friends risking their Friendship, Trust, Love and Hope in search of a legend. A young and privileged teenager with a golden heart, beset with challenges and tribulations we face today with the goal to open many young people's mind with inspirations and hopes that drive them in achieving their dreams. To get out of their comfort zone and finish what they started.", 'score': 0.5322973728179932}
{'title': 'Bionicle: Mask of Light', 'fullplot': "In a land of living machines, two young ones are chosen to seek the legendary Mask of Light to reveal the savior of all the lands from the dark forces of the Makuta. During the course of their adventure, they will call on the heroes of their people, the great Toa. These Toa, masters of nature's forces such as Fire, Wind, Earth & Water, try to protect the chosen ones as they seek their destiny.", 'score': 0.5315042734146118}
{'title': 'Fear No Evil', 'fullplot': 'High school student turns out to be personification of Lucifer. Two arch angels in human form (as women) take him on.', 'score': 0.5295513868331909}
{'title': 'Tales of Vesperia: The First Strike', 'fullplot': 'In a mythical kingdom, the mighty Imperial Knights harness a magical substance known as Aer to power their weapons and protect humanity from the monsters of the forest. But something strange is afoot. The Aer is somehow changing, causing the wilderness to waste away and stirring the woodland beasts to attack with greater frequency. As danger creeps steadily closer to civilization, two young recruits - Flynn, the rigid son of a fallen hero, and the rebellious and brash Yuri - must ride with their fellow Imperial Knights to distant ruins in hopes of uncovering the truth behind the transforming Aer. Some will not survive the thrilling journey. Some will be betrayed. If Flynn and Yuri cannot overcome their differences and learn to fight together, all will be lost for the people of the realm.', 'score': 0.5276793241500854}

On this page