> For the complete MongoDB documentation index, see www.mongodb.com/docs/llms.txt

# MongoDB Vector Search Benchmark Results

This page explores the results of our MongoDB Vector Search [performance benchmark](https://www.mongodb.com/docs/vector-search/benchmark/overview.md#std-label-avs-benchmark-overview). MongoDB offers a range of solutions that support various implementation needs. Customers can choose the levels of performance and accuracy that are suited to their specific business setup.

## Summary of Results

- At 15.3M vectors using `voyage-3-large` embeddings at 2048 dimensions, MongoDB Vector Search with quantization configured retains 90-95% accuracy with \< 50ms query latency.

- Binary quantization offers a cheaper solution, at about one fourth the price of serving the index. Since it rescores with full-fidelity vectors, it can be a preferable option for many large scale workloads, though we recommend running tests on the sample data set to ensure the performance fits your needs.

- We recommend over 1024 dimensions when running larger workloads with quantization.

- Selective filters can improve or worsen performance depending on the value selected for `numCandidates`.

- The additional cost of rescoring for binary quantization shows up in reduced throughput when running highly concurrent workloads.

- Sharding slightly improves throughput, but we still recommend scaling out number of Search Nodes or the number of available cores on a Search Node to improve throughput.

## Recall and Latency Analysis Across Multidimensional Benchmark

The first set of results shows the tests that we ran against a 5.5M document dataset containing multiple dimensionalities of vectors (256, 512, 1024, 2048), all produced using `voyage-3-large`, within each document.

![MongoDB Vector Search Quantization Benchmark Results](/images/performance/quantization.png)

To view the full chart, see the [Claude artifact.](https://claude.ai/public/artifacts/3a204871-ddcd-4010-a522-d8b03c98fdfd)

The scalar quantized results all start at higher levels than the binary quantized results, but stay at their asymptotic level even as `numCandidates` increases. Conversely, binary quantized queries yield more accurate results as more `numCandidates` are requested, approaching the asymptote of scalar quantization, and in some cases passing it, at the cost of higher latency, particularly above `numCandidates` of 1000.

Lower values of `limit` generally require higher `numCandidates` to approach 100% accuracy, because the query is more selective about the very top results. This effect is particularly visible in the binary quantization plot. Higher-dimensional vectors (1024d and above) reach the 90-95% target range at lower `numCandidates`, as the richer representation makes near neighbors easier to distinguish. The tests that `limit` the results to `100` reach the 90-95% target range with lower `numCandidates` than the tests that `limit` the results to `10`, because the larger result set gives the search more opportunities to surface relevant neighbors.

Given this information, we determined that when working with a large dataset, we recommend having dimensionality of at least 1024d and applying quantization to scale than having lower dimensionality and not using quantization, with the amount of vectors requested for the use case playing a factor as well.

## Larger Benchmark Results

For the larger 15.3M vector dataset, we fixed the dimensionality to 2048d and examined the impact of quantization, filtering and concurrency on performance. We chose to pin on 2048d based on the results from the previous set of tests showing that higher dimensions retained recall in a more favorable manner, though 1024d would have likely served just as well to reach the 90-95% recall target.

### Recall and Latency Analysis

We observed that it takes significantly more `numCandidates` when using binary quantization to achieve the 90-95% recall target compared to baseline. Higher `numCandidates` generally means higher latency, but this might vary.

![MongoDB Vector Search Concurrency Benchmark Results](/images/performance/recall-latency.png)

To view the full chart, see the [Claude artifact.](https://claude.ai/public/artifacts/6459f0c6-e3b1-4e8e-99ca-9dde5439a93d)

### Filtering

We observed what happens to recall and latency when using a selective filter on the dataset for ~500k items of the 15.3M items are in the Pet Supplies Category (~3% of the corpus):

![MongoDB Vector Search Filtering Benchmark Results](/images/performance/filtering.png)

To view the full chart, see the [Claude artifact.](https://claude.ai/public/artifacts/10af9b38-3aca-4a1f-ac55-17f151e850dd)

A selective filter is a filter that matches only a small slice of the data (in this case, ~3%). Because the search engine must now find nearest neighbors within this small subset, it has to explore more of the index and evaluate more candidates to reach the same recall. Each query does more work than an unfiltered query.

As anticipated, we can see that the 3% selective filter can cause queries to do more work. For binary quantization at lower `limit` values, this required roughly 4x as much work to achieve 90-95% recall compared to the unfiltered queries.

Future improvements in Lucene 10, which support Acorn-1 search strategies for [Hierarchical Navigable Small Worlds](https://arxiv.org/abs/1603.09320), might improve this process. However, performing ENN (Exact Nearest Neighbor) when the number of requested candidates exceeds the number of vectors matching the metadata filter within a segment demonstrates that filter selectivity plays a large part in query performance, regardless of the selected quantization regime.

### Concurrency

These tests scale concurrent requests between 1, 10, and 100 at the various `limit` values when using scalar and binary quantization. `numCandidates` are selected by choosing values that allow 90-95% recall to be achieved:

![MongoDB Vector Search Concurrency Benchmark Results](/images/performance/concurrency.png)

To view the full chart, see the [Claude artifact.](https://claude.ai/public/artifacts/a60b1ed9-1b87-4806-b6cc-63594def621b)

We observe that scalar quantization achieves higher QPS (queries per second) at all values of `limit`, likely because each query can complete with lower `numCandidates` and without rescoring. At higher concurrencies, the QPS (queries per second) curves for concurrency 10 and concurrency 100 are close together, suggesting that the system is efficiently using available CPU resources, and that extra concurrency primarily affects latency.

One exceptional data point is limit `10`, concurrency `100` for scalar quantization yielding significantly higher QPS. This is likely because no rescoring and lower `limit` values means fewer comparisons are performed for this query, allowing each request to return more quickly and make the cores available to serve other queries.

Scaling out the number of available vCPUs to serve requests, either by scaling up the Search Node tier or scaling up the number of Search Nodes from the minimum of 2 up to 32 nodes might help resolve concurrency bottlenecks and allow you to scale well into the thousands of QPS.

### Sharding

We also observed what would happen if the cluster and collection were sharded (on `_id`) and unfiltered queries were issued against a binary quantized index.

![MongoDB Vector Search Concurrency Benchmark Results](/images/performance/sharding.png)

To view the full chart, see the [Claude artifact.](https://claude.ai/public/artifacts/bc02a204-5741-47a4-a007-2f334ad7624f)

Here, we see that the sharded results have a higher QPS at limit `10` as the lower value of `numCandidates` can be provided to produce results in the 90-95% recall range. This is because the 15.3M dataset is split across three shards, each of which have their own indexes filled with 5.1M vectors spread across segments containing HNSW graphs. We are functionally doing a less advanced search where it is more likely that each query scatter gathered across 3 shards simultaneously could find the closest `n` vectors. For this reason, the QPS is slightly higher when sharding since you can reduce `numCandidates` and have more cores available to serve queries, but the difference is not very significant. Since vector search performance remains reliable, you don't need to shard your cluster to scale its throughput.

**Note:**

The values are similar for limit `100`, numCandidates `200`. We might expect this to perform better for filtered queries with an intelligent shard key matching used as a filter.