Greetings all! I’m using Atlas Vector Search to implement search functionality for my app (following this guide) but I’m running into issues with the filter option on $vectorSearch operator. For context, this is a simplified schema of my data:
{
_id: ObjectId,
owner: ObjectId,
...otherFields
}
and here’s my query:
collection.aggregate(
[
{
"$vectorSearch": {
"queryVector": queryEmbeddings,
"path": "plot_embedding_hf",
"numCandidates": 100,
"limit": limit,
"index": "PlotSemanticSearch",
}
},
]
)
In my app, a user can only search the documents that it owns, so I need to filter by the owner field (type ObjectId). I’d be ideal to use the filter option on $vectorSearch to filter the data before searching, but the docs state that it can only match “boolean, number (not decimals), or string” fields.
Just adding a { $match: { "owner": userId } } after $vectorSearch would not work well because the aggregation would output less documents than the limit if $vectorSearch returns documents owned by other users, which would then get filtered out by $match.
The only solution I could think of would be duplicating ownerId as ownerIdStr but I’d rather not duplicate my data and have to manage another field (and keep that in sync with ownerId).
Are there any solutions to this issue?
Is support for ObjectId on $vectorSearch filter planned (I expect more people could run into a similar problem)?