Expected com.xgen.mongot.index.searchindex, but got instance of com.xgen.mongot.index.lucene.lucenevectorindex

Hi,

I am trying to connect with atlas vector search using python’s langchain library, using the following code -

MONGODB_ATLAS_CLUSTER_URI = os.getenv("MONGODB_CONNECTION_URI")
DB_NAME = os.getenv("MONGODB_DATABASE")
COLLECTION_NAME = collection_name 
# ATLAS_VECTOR_SEARCH_INDEX_NAME = f"{collection_name}_index" 
ATLAS_VECTOR_SEARCH_INDEX_NAME = "indextest1" 

vector_search = MongoDBAtlasVectorSearch.from_connection_string(
MONGODB_ATLAS_CLUSTER_URI,
DB_NAME + "." + COLLECTION_NAME, # type: ignore
OpenAIEmbeddings(openai_api_key=os.getenv("OPENAI_API_KEY"), disallowed_special=()),
index_name=ATLAS_VECTOR_SEARCH_INDEX_NAME,
embedding_key = "plot_embedding",
text_key = "plot",
)

vector_chain = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=vector_search.as_retriever(), verbose=True)

vector_chain.run("give me the plot of the 1914 movie - The Perils of Pauline")

I am using -
langchain version - 0.0.188
pymongo version - 4.6.1
python version - 3.9.0

the above code is throwing this error -

Expected com.xgen.mongot.index.SearchIndex, but got instance of com.xgen.mongot.index.lucene.LuceneVectorIndex, full error: {'ok': 0.0, 'errmsg': 'Expected com.xgen.mongot.index.SearchIndex, but got instance of com.xgen.mongot.index.lucene.LuceneVectorIndex', 'code': 8, 'codeName': 'UnknownError', '$clusterTime': {'clusterTime': Timestamp(1706254457, 30), 'signature': {'hash': b'\x92\x0f>\x12\x12p\xa3\xf3L\x9f\xe7j!\xf1\xc6D\xadY;\t', 'keyId': 7279102472952479745}}, 'operationTime': Timestamp(1706254457, 30)}

The catch is, this same code piece was working perfectly fine, exactly as intended, nothing has been changed but got this error

This is how my index looks like -

{
  "fields": [
    {
      "numDimensions": 1536,
      "path": "plot_embedding",
      "similarity": "cosine",
      "type": "vector"
    },
    {
      "path": "genres",
      "type": "filter"
    },
    {
      "path": "year",
      "type": "filter"
    }
  ]
}

Need urgent help with this,
ThanksThis text will be hidden

Hi @Yuvraj_Singh1, I tried running your code setup with the specific versions of various packages and didn’t run into the above error. Did you retry and still get this error?
Any reason for using the langchain version 0.0.188 instead of the more latest ones?
If the problem still exits for you can we please get on a quick zoom call / screen share? My email is “prakul.agarwal”[at]mongodb[dot]com and we can coordinate a time

Hi @Yuvraj_Singh1 ,

I have encountered this issue as well, and I believe I’ve figured out what is happening here. In Atlas Search, there are two types of indexes: vectorSearch and search. It’s possible that you’ve created a vectorSearch index and are using the $search operator in your aggregation pipeline. This would trigger a reminder about the different types of indexes.

Operator $search is for search type of index and $vectorSearch is for vectorSearch type of index. Please check your index type as followings: