PlanExecutor error during aggregation

Hi guys, I am using MongoDB atlas and I am facing the issue of filtering the documents before doing vector search. I made two codes but none of them worked.

from pymongo import MongoClient
from langchain.vectorstores.mongodb_atlas import MongoDBAtlasVectorSearch
from langchain.embeddings import OpenAIEmbeddings

client_db = MongoClient(CONNECTION_STRING)
db = client_db[“main”]
collection = db[“data”]

embedding_model = OpenAIEmbeddings(api_key=openai_key)

vector_search = MongoDBAtlasVectorSearch(
collection,
embedding_model,
index_name=“vector_index”,
text_key=“description_embedding”,
relevance_score_fn=“cosine”

)

query = “Show me some properties for sale at Accra, Ghana with 4 beds”

Get relevant documents using vector search

search_results = vector_search.similarity_search(query, k=5, pre_filter={ “$and”: [{ “price”: { “$lt”: 20000 }}]})

I am getting error:
OperationFailure: PlanExecutor error during aggregation :: caused by :: Path ‘price’ needs to be indexed as number, full error: {‘ok’: 0.0, ‘errmsg’: “PlanExecutor error during aggregation :: caused by :: Path ‘price’ needs to be indexed as number”, ‘code’: 8, ‘codeName’: ‘UnknownError’, ‘$clusterTime’: {‘clusterTime’: Timestamp(1712542957, 1), ‘signature’: {‘hash’: b’\xbfb\xbb\xba\x84!\xa1\x05@\x8fXF\xe5\x82X_\x0ea\x88\xab’, ‘keyId’: 7349586743160471558}}, ‘operationTime’: Timestamp(1712542957, 1)}

Other code block:
import pymongo
import datetime
query = “Show me some properties for sale at Accra, Ghana with 4 beds”
client_embedding = OpenAIEmbeddings(api_key=openai_key)
query_embedding = client_embedding.embed_query(query)

connect to your Atlas cluster

client_db = MongoClient(CONNECTION_STRING)
db = client_db[“rehaniAI”]
collection = db[“data”]

define pipeline

pipeline=[{
“$vectorSearch”: {
“index”: “vector_index”,
“path”: “embedding”,
“filter”: {
“$and”: [
{
“price”: {
“$gt”: 0
},
},
{
“price”: {
“$lt”: 20000
}
}
]
},
“queryVector”: query_embedding,
“numCandidates”: 150,
“limit”: 10,
}
}]

run pipeline

result = collection.aggregate(pipeline)

print results

for i in result:
print(I)

still the same error
OperationFailure: PlanExecutor error during aggregation :: caused by :: Path ‘price’ needs to be indexed as number, full error: {‘ok’: 0.0, ‘errmsg’: “PlanExecutor error during aggregation :: caused by :: Path ‘price’ needs to be indexed as number”, ‘code’: 8, ‘codeName’: ‘UnknownError’, ‘$clusterTime’: {‘clusterTime’: Timestamp(1712542705, 1), ‘signature’: {‘hash’: b’\x15\xd5\xa8\x1d\x9d\x1f\x97\xe8\xc5\xef\xc2\xc7\xb9"\x11y\x89`\xd6K’, ‘keyId’: 7349586743160471558}}, ‘operationTime’: Timestamp(1712542705, 1)

I created the index named price for the price attribute as ascending order still same issue. Any help would be much appreciated.

Thanks
Best Regards,
Owais Ahmed

Hi Owais Ahmed,

Most probably the problem lies in your Vector Search Index definition. The error is clear: price should be indexed as a number to be used with $lt , as you can see in the error:

So go to the index definition and change that field to be indexed as a number.

Diego

2 Likes

Hi @Diego_Freniche


I have made the index and the type of index is selected as (1) asc . How can change thefield to be indexed as a number.

I appreciate your time and reply,
Best Regards,
Owais Ahmed

Hi @Owais_Ahmed1,

You’re sharing a screenshot of the database indexes that you see when you 1) select a collection, 2) click on Indexes. To make Vector Search work you need to create a Vector Search Index, which is a Lucene Index (different from the regular database indexes).

To do so, click on the Atlas Search tab. If the Index is there, show it a JSON and share the configuration. If it’s not there, create it!

You have step by step instructions in the documentation.

Best Regards,

Diego

Hi Diego,

I got the same issue. it’s not because there is no vector search index (note that there is already a vector search index called “vector_index” in this case), it’s because additional field ('price" in this case) that requires some kind of token to be specified in the filter of @vectorSearch . In the case, a text index was created, but apparently it didn’t work. Do you mean a second vector search index against the “price” has to be created? It’s very confusiing …

Thank you for your help,
Jeremy

H guys,
What i want to achieve here is that first i want to pre filter records on this basis of range of price attribute (float type) then perform vector search on those filtered records. I have attribute named embedding which is vector value and description_embedding attribute as the content of that vector.

@Diego_Freniche @Jeremy_Xiang Any help here would be a great help to me.


I have also defined the price index on price attribute in asc order and sparse index as their are values in the price attribute which can be null. Still I am getting error
PlanExecutor error during aggregation :: caused by :: Path ‘price’ needs to be indexed as number, full error: {‘ok’: 0.0, ‘errmsg’: “PlanExecutor error during aggregation :: caused by :: Path ‘price’ needs to be indexed as number”, ‘code’: 8, ‘codeName’: ‘UnknownError’, ‘$clusterTime’: {‘clusterTime’: Timestamp(1713194108, 13), ‘signature’: {‘hash’: b’\xab-c\r\xc1\xd8\x84\x9c\x91!\x8a(\xd2Snq\x88\xc7\x9a\x1c’, ‘keyId’: 7349586743160471558}}, ‘operationTime’: Timestamp(1713194108, 13)}
I tried the steps mentioned here at : https://www.mongodb.com/docs/atlas/atlas-search/field-types/number-type/#define-the-index-for-the-fts-field-type-type
but wasn’t able to find the mentioned options in the URL in MongoDB Compass.