注意
本教程使用 LangChain 的Python库。有关使用 JavaScript 库的教程,请参阅开始使用 LangChain JavaScript/ Typescript 集成。
您可以将MongoDB Vector Search 与 LangChain 集成,以构建LLM 应用程序并实现检索增强生成 (RAG)。本教程演示如何开始使用MongoDB Vector Search 和 LangChain 对数据执行语义搜索并构建RAG实施。具体来说,您执行以下操作:
设置环境。
在MongoDB中存储自定义数据。
对数据创建MongoDB Vector Search索引。
运行以下向量搜索查询:
语义搜索。
带分数的语义搜索。
带元数据预过滤的语义搜索。
使用 MongoDB Vector Search 来回答有关数据的问题,从而实施RAG。
背景
LangChain 是一个开源框架,它通过使用“链”简化了 LLM 应用程序的创建。链是特定于 LangChain 的组件,可以组合用于各种 AI 使用案例,包括 RAG。
By integrating MongoDB Vector Search with LangChain, you can use MongoDB as a vector database and use MongoDB Vector Search to implement RAG by retrieving semantically similar documents from your data. To learn more about RAG, see Retrieval-Augmented Generation (RAG) with MongoDB.
先决条件
如要完成本教程,您必须具备以下条件:
以下MongoDB 集群类型之一:
使用 Python 和 Docker 创建的本地 Atlas 部署。安装
atlas-local-lib-py(pip install atlas-local-lib-py),以编程方式创建和管理本地部署。要了解更多信息,请参阅 atlas-local-lib-py 存储库。A MongoDB Community cluster with Search and Vector Search installed.
A Voyage AI API key. To create an API key, see Manage Voyage AI Model API Keys.
OpenAI API密钥。您必须拥有一个具有可用于API请求的积分的 OpenAI 帐户。要学习;了解有关注册 OpenAI 帐户的更多信息,请参阅 OpenAI API网站。
运行交互式Python笔记本(例如 Colab)的环境。
设置环境
为此教程设置环境。 通过保存扩展名为 .ipynb 的文件来创建交互式Python笔记本。 此 Notebook 允许您单独运行Python代码片段,并且您将使用它来运行本教程中的代码。
要设立笔记本环境,请执行以下操作:
安装并导入依赖项。
运行以下命令:
pip install --quiet --upgrade langchain langchain-community langchain-core langchain-mongodb langchain-voyageai langchain-openai langchain-text-splitters pymongo pypdf
然后,运行以下代码以导入所需的包:
import os, pymongo, pprint from langchain_community.document_loaders import PyPDFLoader from langchain_core.output_parsers import StrOutputParser from langchain_core.runnables import RunnablePassthrough from langchain_mongodb import MongoDBAtlasVectorSearch from langchain_voyageai import VoyageAIEmbeddings from langchain_openai import ChatOpenAI from langchain.prompts import PromptTemplate from langchain_text_splitters import RecursiveCharacterTextSplitter from pymongo import MongoClient from pymongo.operations import SearchIndexModel
定义环境变量。
运行以下代码,将占位符替换为以下值:
您的 Voyage AI和 OpenAI API密钥。
MongoDB集群的连接字符串。
os.environ["VOYAGE_API_KEY"] = "<voyage-api-key>" os.environ["OPENAI_API_KEY"] = "<openai-api-key>" MONGODB_URI = "<connection-string>"
注意
将 <connection-string> 替换为您的 Atlas 集群或本地部署的连接字符串。
连接字符串应使用以下格式:
mongodb+srv://<db_username>:<db_password>@<clusterName>.<hostname>.mongodb.net
重要: 确保连接字符串包含数据库用户的凭证。要学习有关查找连接字符串的更多信息,请参阅通过客户端库连接到集群。
使用MongoDB作为向量存储
然后,将自定义数据加载到MongoDB中,并将MongoDB 集群实例化为向量数据库(也称为向量存储)。将以下代码片段复制并粘贴到笔记本中。
加载示例数据。
For this tutorial, you use a publicly accessible PDF document about a recent MongoDB earnings report as the data source for your vector store.
要加载样本数据,请运行以下代码片段。 它执行以下操作:
从指定的 URL 检索 PDF 并加载原始文本数据。
使用文本拆分器将数据分割为较小的文档。
指定数据块参数,它确定每个文档中的字符数以及两个连续文档之间应重叠的字符数。
# Load the PDF loader = PyPDFLoader("https://investors.mongodb.com/node/13176/pdf") data = loader.load() # Split PDF into documents text_splitter = RecursiveCharacterTextSplitter(chunk_size=200, chunk_overlap=20) docs = text_splitter.split_documents(data) # Print the first document docs[0]
Document(metadata={'producer': 'West Corporation using ABCpdf', 'creator': 'PyPDF', 'creationdate': '2025-03-05T21:06:26+00:00', 'title': 'MongoDB, Inc. Announces Fourth Quarter and Full Year Fiscal 2025 Financial Results', 'source': 'https://investors.mongodb.com/node/13176/pdf', 'total_pages': 9, 'page': 0, 'page_label': '1'}, page_content='MongoDB, Inc. Announces Fourth Quarter and Full Year Fiscal 2025 Financial Results\nMarch 5, 2025\nFourth Quarter Fiscal 2025 Total Revenue of $548.4 million, up 20% Year-over-Year')
实例化向量存储。
运行以下代码以从示例文档中创建一个名为 vector_store 的向量存储实例。此代码段指定如下内容:
MongoDB 集群的连接字符串。
langchain_db.test作为存储文档的MongoDB命名空间。来自 Voyage AI的
voyage-3-large嵌入模型。将文本转换为embedding字段的向量嵌入。vector_index作为用于查询向量存储的索引。
# Instantiate the vector store using your MongoDB connection string vector_store = MongoDBAtlasVectorSearch.from_connection_string( connection_string = MONGODB_URI, namespace = "langchain_db.test", embedding = VoyageAIEmbeddings(model="voyage-3-large"), index_name = "vector_index" ) # Add documents to the vector store vector_store.add_documents(documents=docs)
运行示例代码后,如果您使用的是Atlas ,则可以导航到Atlas 用户界面中的 langchain_db.test命名空间来验证向量嵌入。
创建MongoDB Vector Search 索引
要在向量存储上启用向量搜索查询,请使用 LangChain 辅助工具方法或 PyMongo 驱动程序方法在 langchain_db.test 集合上创建 MongoDB 向量搜索索引。
在笔记本中,按照首选方法运行以下代码。此索引定义指定对以下字段进行索引:
embeddingfield as the vector type. Theembeddingfield contains the embeddings created using Voyage AI'svoyage-3-largeembedding model. The index definition specifies1024vector dimensions and measures similarity usingcosine.page_labelfield as the filter type for pre-filtering data by the page number in the PDF.
# Use helper method to create the vector search index vector_store.create_vector_search_index( dimensions = 1024, # The number of vector dimensions to index filters = [ "page_label" ] )
# Connect to your cluster client = MongoClient(MONGODB_URI) collection = client["langchain_db"]["test"] # Create your vector search index model, then create the index search_index_model = SearchIndexModel( definition={ "fields": [ { "type": "vector", "path": "embedding", "numDimensions": 1024, "similarity": "cosine" }, { "type": "filter", "path": "page_label" } ] }, name="vector_index", type="vectorSearch" ) collection.create_search_index(model=search_index_model)
The index should take about one minute to build. While it builds, the index is in an initial sync state. When it finishes building, you can start querying the data in your collection.
运行向量搜索查询
MongoDB构建索引后,即可对数据运行向量搜索查询。以下示例演示了可以对矢量化数据运行的各种查询。
以下查询使用similarity_search方法对字符串MongoDB acquisition执行基本语义搜索。 它返回按相关性排名的文档列表。
query = "MongoDB acquisition" results = vector_store.similarity_search(query) pprint.pprint(results)
[Document(id='67f0259b8bb2babc06924409', metadata={ ... }, page_content='SOURCE MongoDB, Inc.'), Document(id='67f0259b8bb2babc0692432f', metadata={ ... }, page_content='MongoDB platform. In fiscal year 2026 we expect to see stable consumption growth in Atlas, our main growth driver," said Dev Ittycheria, President\nand Chief Executive Officer of MongoDB .'), Document(id='67f0259b8bb2babc06924355', metadata={ ... }, page_content='conjunction with the acquisition of Voyage, MongoDB is announcing a stock buyback program of $200 million, to offset the\ndilutive impact of the acquisition consideration.'), Document(id='67f0259b8bb2babc069243a6', metadata={ ... }, page_content="MongoDB's unified, intelligent data platform was built to power the next generation of applications, and MongoDB is the most widely available, globally")]
以下查询使用 similarity_search_with_score 方法对字符串 MongoDB acquisition 执行语义搜索,并指定 k 参数以将返回的文档数量限制为 3 。
注意
The k parameter in this example refers to the similarity_search_with_score method option, not the knnBeta operator option of the same name.
It returns the three most relevant documents and a relevance score between 0 and 1.
query = "MongoDB acquisition" results = vector_store.similarity_search_with_score( query = query, k = 3 ) pprint.pprint(results)
[(Document(id='67f0259b8bb2babc06924409', metadata={ ... }, page_content='SOURCE MongoDB, Inc.'), 0.8193451166152954), (Document(id='67f0259b8bb2babc0692432f', metadata={ ... }, page_content='MongoDB platform. In fiscal year 2026 we expect to see stable consumption growth in Atlas, our main growth driver," said Dev Ittycheria, President\nand Chief Executive Officer of MongoDB .'), 0.7815237045288086), (Document(id='67f0259b8bb2babc06924355', metadata={ ... }, page_content='conjunction with the acquisition of Voyage, MongoDB is announcing a stock buyback program of $200 million, to offset the\ndilutive impact of the acquisition consideration.'), 0.7788857221603394)]
You can pre-filter your data by using an MQL match expression that compares the indexed field with another value in your collection. You must index any metadata fields that you want to filter by as the filter type. To learn more, see How to Index Fields for Vector Search.
以下查询使用 similarity_search_with_score 方法对字符串 MongoDB acquisition 执行语义搜索。它还指定了以下内容:
k参数,用于将要返回的文档数量限制为3。A pre-filter on the
page_labelfield that uses the$eqoperator to match documents appearing on page 2 only.
It returns the three most relevant documents from page 2 and a relevance score between 0 and 1.
query = "MongoDB acquisition" results = vector_store.similarity_search_with_score( query = query, k = 3, pre_filter = { "page_label": { "$eq": 2 } } ) pprint.pprint(results)
[(Document(id='67f0259b8bb2babc06924355', metadata={ ... 'page_label': '2'}, page_content='conjunction with the acquisition of Voyage, MongoDB is announcing a stock buyback program of $200 million, to offset the\ndilutive impact of the acquisition consideration.'), 0.7788857221603394), (Document(id='67f0259b8bb2babc06924351', metadata={ ... 'page_label': '2'}, page_content='Measures."\nFourth Quarter Fiscal 2025 and Recent Business Highlights\nMongoDB acquired Voyage AI, a pioneer in state-of-the-art embedding and reranking models that power next-generation'), 0.7606035470962524), (Document(id='67f0259b8bb2babc06924354', metadata={ ... 'page_label': '2'}, page_content='data.\nMongoDB completed the redemption of 2026 Convertible Notes, eliminating all debt from the balance sheet. Additionally, in'), 0.7583936452865601)]
提示
有关语义搜索方法的完整列表,请参阅API参考。
回答有关数据的问题
本部分演示如何使用MongoDB Vector Search 和 LangChain 在应用程序中实现RAG。现在您已经使用MongoDB Vector Search检索语义相似的文档,运行以下代码示例以提示法学硕士回答基于这些文档的问题。
此示例执行以下操作:
定义 LangChain 提示模板,指示 LLM 使用这些文档作为查询的上下文。LangChain 将这些文档传递给
{context}输入变量,并将您的查询传递给{question}变量。使用样本查询调用链。
返回 LLM 的响应和作为上下文使用的文档。 产生的响应可能会有所不同。
# Instantiate MongoDB Vector Search as a retriever retriever = vector_store.as_retriever( search_type = "similarity", search_kwargs = { "k": 10 } ) # Define a prompt template template = """ Use the following pieces of context to answer the question at the end. {context} Question: {question} """ prompt = PromptTemplate.from_template(template) model = ChatOpenAI(model="gpt-4o") # Construct a chain to answer questions on your data chain = ( { "context": retriever, "question": RunnablePassthrough()} | prompt | model | StrOutputParser() ) # Prompt the chain question = "What was MongoDB's latest acquisition?" answer = chain.invoke(question) print("Question: " + question) print("Answer: " + answer) # Return source documents documents = retriever.invoke(question) print("\nSource documents:") pprint.pprint(documents)
Question: What was MongoDB's latest acquisition? Answer: MongoDB's latest acquisition was Voyage AI, a pioneer in state-of-the-art embedding and reranking models. Source documents: [Document(id='67f0259b8bb2babc06924409', metadata={'_id': '67f0259b8bb2babc06924409', ... 'page_label': '9'}, page_content='SOURCE MongoDB, Inc.'), Document(id='67f0259b8bb2babc06924351', metadata={'_id': '67f0259b8bb2babc06924351', ... 'page_label': '2'}, page_content='Measures."\nFourth Quarter Fiscal 2025 and Recent Business Highlights\nMongoDB acquired Voyage AI, a pioneer in state-of-the-art embedding and reranking models that power next-generation'), Document(id='67f0259b8bb2babc0692432f', metadata={'_id': '67f0259b8bb2babc0692432f', ... 'page_label': '1'}, page_content='MongoDB platform. In fiscal year 2026 we expect to see stable consumption growth in Atlas, our main growth driver," said Dev Ittycheria, President\nand Chief Executive Officer of MongoDB .'), Document(id='67f0259b8bb2babc06924355', metadata={'_id': '67f0259b8bb2babc06924355', ... 'page_label': '2'}, page_content='conjunction with the acquisition of Voyage, MongoDB is announcing a stock buyback program of $200 million, to offset the\ndilutive impact of the acquisition consideration.'), Document(id='67f0259b8bb2babc069243a6', metadata={'_id': '67f0259b8bb2babc069243a6', ... 'page_label': '4'}, page_content="MongoDB's unified, intelligent data platform was built to power the next generation of applications, and MongoDB is the most widely available, globally"), Document(id='67f0259b8bb2babc06924329', metadata={'_id': '67f0259b8bb2babc06924329', ... 'page_label': '1'}, page_content='MongoDB, Inc. Announces Fourth Quarter and Full Year Fiscal 2025 Financial Results\nMarch 5, 2025\nFourth Quarter Fiscal 2025 Total Revenue of $548.4 million, up 20% Year-over-Year'), Document(id='67f0259b8bb2babc069243a7', metadata={'_id': '67f0259b8bb2babc069243a7', ... 'page_label': '4'}, page_content='distributed database on the market. With integrated capabilities for operational data, search, real-time analytics, and AI-powered retrieval, MongoDB'), Document(id='67f0259b8bb2babc069243a5', metadata={'_id': '67f0259b8bb2babc069243a5', ... 'page_label': '4'}, page_content="Headquartered in New York, MongoDB's mission is to empower innovators to create, transform, and disrupt industries with software and data."), Document(id='67f0259b8bb2babc06924354', metadata={'_id': '67f0259b8bb2babc06924354', ... 'page_label': '2'}, page_content='data.\nMongoDB completed the redemption of 2026 Convertible Notes, eliminating all debt from the balance sheet. Additionally, in'), Document(id='67f0259b8bb2babc069243a9', metadata={'_id': '67f0259b8bb2babc069243a9', ... 'page_label': '4'}, page_content='50,000 customers across almost every industry—including 70% of the Fortune 100—rely on MongoDB for their most important applications. To learn\nmore, visit mongodb.com .\nInvestor Relations')]
此示例执行以下操作:
将 MongoDB Vector Search 实例化为检索器以查询相似文档,包括以下可选参数:
k用于仅搜索10个最相关的文档。score_threshold仅使用相关度分数高于0.75的文档。注意
This parameter refers to a relevance score that Langchain uses to normalize your results, and not the relevance score used in MongoDB Search queries. To use MongoDB Search scores in your RAG implementation, define a custom retriever that uses the
similarity_search_with_scoremethod and filters by the MongoDB Search score.pre_filter在page_label字段中过滤仅出现在 2 页上的文档。
定义 LangChain 提示模板,指示 LLM 使用这些文档作为查询的上下文。LangChain 将这些文档传递给
{context}输入变量,并将您的查询传递给{question}变量。使用样本查询调用链。
返回 LLM 的响应和作为上下文使用的文档。 产生的响应可能会有所不同。
# Instantiate MongoDB Vector Search as a retriever retriever = vector_store.as_retriever( search_type = "similarity", search_kwargs = { "k": 10, "score_threshold": 0.75, "pre_filter": { "page_label": { "$eq": 2 } } } ) # Define a prompt template template = """ Use the following pieces of context to answer the question at the end. {context} Question: {question} """ prompt = PromptTemplate.from_template(template) model = ChatOpenAI(model="gpt-4o") # Construct a chain to answer questions on your data chain = ( { "context": retriever, "question": RunnablePassthrough()} | prompt | model | StrOutputParser() ) # Prompt the chain question = "What was MongoDB's latest acquisition?" answer = rag_chain.invoke(question) print("Question: " + question) print("Answer: " + answer) # Return source documents documents = retriever.invoke(question) print("\nSource documents:") pprint.pprint(documents)
Question: What was MongoDB's latest acquisition? Answer: MongoDB's latest acquisition was Voyage AI, a pioneer in state-of-the-art embedding and reranking models. Source documents: [Document(id='67f0259b8bb2babc06924351', metadata={'_id': '67f0259b8bb2babc06924351', ... 'page_label': '2'}, page_content='Measures."\nFourth Quarter Fiscal 2025 and Recent Business Highlights\nMongoDB acquired Voyage AI, a pioneer in state-of-the-art embedding and reranking models that power next-generation'), Document(id='67f0259b8bb2babc06924355', metadata={'_id': '67f0259b8bb2babc06924355', ... 'page_label': '2'}, page_content='conjunction with the acquisition of Voyage, MongoDB is announcing a stock buyback program of $200 million, to offset the\ndilutive impact of the acquisition consideration.'), Document(id='67f0259b8bb2babc06924354', metadata={'_id': '67f0259b8bb2babc06924354', ... 'page_label': '2'}, page_content='data.\nMongoDB completed the redemption of 2026 Convertible Notes, eliminating all debt from the balance sheet. Additionally, in'), Document(id='67f0259b8bb2babc06924358', metadata={'_id': '67f0259b8bb2babc06924358', ... 'page_label': '2'}, page_content='Lombard Odier, a Swiss private bank, partnered with MongoDB to migrate and modernize its legacy banking technology'), Document(id='67f0259b8bb2babc06924352', metadata={'_id': '67f0259b8bb2babc06924352', ... 'page_label': '2'}, page_content="AI applications. Integrating Voyage AI's technology with MongoDB will enable organizations to easily build trustworthy,"), Document(id='67f0259b8bb2babc0692435a', metadata={'_id': '67f0259b8bb2babc0692435a', ... 'page_label': '2'}, page_content='applications from a legacy relational database to MongoDB 20 times faster than previous migrations.\nFirst Quarter and Full Year Fiscal 2026 Guidance'), Document(id='67f0259b8bb2babc06924356', metadata={'_id': '67f0259b8bb2babc06924356', ... 'page_label': '2'}, page_content='For the third consecutive year, MongoDB was named a Leader in the 2024 Gartner® Magic Quadrant™ for Cloud'), Document(id='67f0259b8bb2babc0692434d', metadata={'_id': '67f0259b8bb2babc0692434d', ... 'page_label': '2'}, page_content='compared to $121.5 million of cash from operations in the year-ago period. MongoDB used $29.6 million of cash in capital'), Document(id='67f0259b8bb2babc0692434c', metadata={'_id': '67f0259b8bb2babc0692434c', ... 'page_label': '2'}, page_content='Cash Flow: During the year ended January 31, 2025, MongoDB generated $150.2 million of cash from operations,'), Document(id='67f0259b8bb2babc06924364', metadata={'_id': '67f0259b8bb2babc06924364', ... 'page_label': '2'}, page_content='MongoDB will host a conference call today, March 5, 2025, at 5:00 p.m. (Eastern Time) to discuss its financial results and business outlook. A live')]
通过观看学习
请观看本视频教程,以了解更多关于使用 LangChain 和 MongoDB 进行语义搜索和检索增强生成 (RAG) 的信息。
时长:8 分钟