$vectorSearch is not allowed

MongoDB Atlas Vector Search
,
1

Hello,

since today I’m getting this error message: “$vectorSearch is not allowed or the syntax is incorrect, see the Atlas documentation for more information”

and this is my function that get called.

   const client = new MongoClient(this.configService.get('MONGO_LINK') || '');
    const namespace = 'sample_mflix.embedded_movies';
    const [dbName, collectionName] = namespace.split('.');
    const collection = client.db(dbName).collection(collectionName);

    const vectorStore = new MongoDBAtlasVectorSearch(
      new OpenAIEmbeddings({
        openAIApiKey: this.configService.get('OPENAI_API_KEY'),
      }),
      {
        collection,
        indexName: 'default', // The name of the Atlas search index. Defaults to "default"
        textKey: 'plot', // The name of the collection field containing the raw content. Defaults to "text"
        embeddingKey: 'plot_embedding', // The name of the collection field containing the embedded text. Defaults to "embedding"
      },
    );

    const resultOne: any = await vectorStore.similaritySearch(question, 4);
    console.log(resultOne);

And this is my search index:

{
  "mappings": {
    "fields": {
      "plot_embedding": [
        {
          "dimensions": 1536,
          "similarity": "cosine",
          "type": "knnVector"
        }
      ]
    }
  }
}

I use langchain 0.0.166

What do I missing?

3 Likes
2

I have the same problem however my code worked a few weeks ago. I also use MongoDBAtlasVectorSearch from LangChain. I did some research and found out that even direct querying from Atlas UI page using $vectorSearch doesn’t work anymore.

My search index looks like this

{
    "mappings": {
        "dynamic": false,
        "fields": {
            "embedding": {
                "dimensions": 1536,
                "similarity": "cosine",
                "type": "knnVector"
            },
            "location": {
                "type": "geo"
            }
        }
    }
}

There is also a geo-index part here but it doesn’t matter. Here is the search query:

{
    "$vectorSearch": {
      "index": "vector_geo_index",
      "path": "embedding",
      "numCandidates": 10,
      "limit": 2,
      "queryVector": [...]
    }
}

Error: Failed to run this query. Reason: $vectorSearch is not allowed or the syntax is incorrect, see the Atlas documentation for more information

This did work previously. Also, the old (deprecated in favor of $vectorSearch) knnBeta operator works fine:

{
    "$search": {
      "index": "vector_geo_index",
      "knnBeta": {
        "vector": [...],
        "path": "embedding",
        "k": 2
      }
    }
}

My guess it is not related to Langchain. It is probably something on the Atlas side. Docs say that the vector search is a preview feature and not recommended for prod. Maybe it is not very stable at the moment.

3

Hey Alexey_Zelenovsky, I am also facing the same issue. Not able to perform vector search with mongoDB using langchain documentation and even MongoDBs own documentation . Can you suggest some other ways using which I can store the vector embeddings in database and perform a vector search on same.

4

@Deepak_Dev, I decided not to use MongoDBAtlasVectorSearch because there is no way I can switch it from $vectorSearch to $search with the knnBeta operator.

I replaced this

db = client[database_name]
my_collection = db["collection_name"]
embeddings = OpenAIEmbeddings()
vectorstore = MongoDBAtlasVectorSearch(my_collection, embeddings, index_name="my_search_index")
search_result = self.vectorstore.similarity_search(query="tacos", k=2)

with something like this:

db = client[database_name]
my_collection = db["collection_name"]
embeddings = OpenAIEmbeddings()
mongo_query = [
            {
                "$search": {
                    "index": "vector_geo_index",
                    "knnBeta": {
                        "vector": embeddings.embed_query('tacos'),
                        "path": "embedding",
                        "k": 2,
                    }
                }
            }
        ]

search_result = my_collection.aggregate(mongo_query);

This code does the same as vectorstore.similarity_search internally.
I don’t know if I’m allowed to share a link to my Github here. DM me if you want to see the whole implementation.

5

@Alexey_Zelenovsky , You can share your github link over here only. As In this community I can’t DM you or you can tell me a way through which I can connect with you and get the the github link. And one more thing, KnnBeta operator is deprecated now. according to below mentioned documentation.

https://www.mongodb.com/docs/atlas/atlas-search/knn-beta/#:~:text=You%20can%20run%20kNN%20queries,than%20the%20value%20for%20%24limit%20.

6

I’ve kind of “solved” it for me by using an older version of langchain.

This problem occurs since version "langchain": "^0.0.165",
So just use version “langchain”: “^0.0.164”,

7

Hi everybody, I think it might not be related to LangChain, as I tried the $vectorSearch tutorial provided my MongoDB using their own movies dataset, and I’m facing the exact same issue.

1 Like
8

Same here, @Victor_Carrara , It is not langchain issue.

9

It is true that knnBeta is deprecated in favor of $vectorSearch. However, the $vectorSearch itself is a Preview feature which means it is not stable. So switching to the knnBeta operator is a temporary solution until the $vectorSearch starts working properly.

Here is my implementation on Github of the vector search with knnBeta if anybody is looking for an example.

1 Like
10

I’m not using LangChain, but running into this problem as well. Deeply disappointing, with all the “you don’t need a vector database” messaging Mongo’s been using.

11

Hello all, apologies for the error and for the delayed response. To clarify, the error that you’re seeing here “$vectorSearch is not allowed or the syntax is incorrect, see the Atlas documentation for more information” is due to an issue in M0-M5 Clusters which will be resolved after our deployment tomorrow, it is not related to LangChain.

The origin of this error stems from a slight difference in how our shared tier clusters work when compared to dedicated tier, hence clusters M10 and above are not affected.

To provide a bit more context, $vectorSearch is our target interface for Vector Search in the long term, and as such we’ve updated LangChain to utilize this new syntax. That said, we do not currently have plans to remove support in Atlas for knnBeta, and will not do so until every Cluster in Atlas is running a version that supports $vectorSearch. We will provide additional guidance ahead of any changes to support for knnBeta.

And one last clarification, the entire Vector Search service is in Preview, whether it be through $vectorSearch or knnBeta.

Finally, for anyone who would like to chat in more depth please feel free to follow up here or reach out to me directly at Benjamin.flast@mongodb.com.

4 Likes
12

Thank you so much, Benjamin!
I’d much prefer to keep my company’s vector search needs here with our preferred vendor, so a near-future resolution of this issue is very good news.
(BTW, our production cluster is M10, but our dev cluster, where I was working when I encountered the error, is on a shared tier.)

13

Hey Benjamin! thanks for the update. I still have the problem and before to restart everything, I would like to know if the fix has been deployed?
if so, how can i know / check if my cluster has been updated?

Thanks

2 Likes
14

Hey Benjamin, do you know when this $vectorSearch gonna be released? It has been a blocker for our project. We have vector DB in pinecone but were planning to move to MongoDB for vector search capability and now we are basically stuck. Would really appreciate it if you let us know the timeline.

This was hyped but now has been a real pain for developers like us who trusted the MongoDB ecosystem. Would appreciate it if Mongo could communicate it better.

15

Hello All,

Just to provide a quick update, the update has been made, it’s just making it through all of the testing environments. This should be available on shared tier clusters (M0-M5) by the end of day Monday.

As a quick reminder, this can be accessed on dedicated cluster M10 and above today. Or you can continue to query utilizing the knnBeta operator until Monday.

2 Likes
16

Hi Everyone!

Thank you all for your patience, this issue is now resolved. You should be able to use the $vectorSearch stage on shared tier clusters (M0-M5) running on MongoDB versions 6.0.11 and 7.0.2.

17

I am still getting this same error. I am on version 7.0.2 M30. What am I missing? Your previous message 5 days ago says M10 and above can access today. This is massively frustrating …

18

You’ll have to upgrade to 7.0.2

19

I just updated my message please review it.

20

It is working for me now