Docs Menu

Docs HomeLaunch & Manage MongoDBMongoDB Atlas

Atlas Search Index Performance

On this page

  • Resource Requirements
  • Index Size and Configuration
  • Considerations
  • Creating and Updating an Atlas Search Index
  • Eventual Consistency and Indexing Latency
  • Document Mapping Explosions
  • Storing Source Fields
  • Scaling Considerations
  • Atlas Search Upgrade
  • Scaling Up Indexing Performance
  • Atlas Cluster Configuration Change

Important

If you create an Atlas Search index for a collection that has or will soon have more than two billion documents, you must shard your cluster.

When you create an Atlas Search index, field mapping defaults to dynamic, which means that Atlas Search dynamically indexes all the datatypes that can be dynamically indexed in your collection. Other options such as enabling highlights can also result in your index taking up more disk space. You can reduce the size and performance footprint of your Atlas Search index by:

  • Specifying a custom index definition to narrow the amount and type of data that is indexed.

  • Setting the store option to false when specifying a string type in an index definition.

Note

Some limitations apply to Atlas Search on M0, M2, and M5 clusters only. To learn more, see Atlas Search Free and Shared Tier Limitations.

Some index configuration options can lead to indexes that take up a significant proportion of your disk space. In some cases, your index could be many times larger than the size of your data. Although this is expected behavior, it's important to be aware of the following indexing-intensive features:

The Atlas Search autocomplete operator can be used to build functionality similar to search-as-you-type in your application. The Atlas Search autocomplete field type can cause large indexes, especially in the following cases:

  • Using nGram tokenization.

  • Setting a wide minGrams to maxGrams range.

  • Setting a minGram value of 1 on a collection with millions of documents.

You can reduce the space used by the autocomplete type index by doing the following:

  • Reduce the range of minGrams and maxGrams to the minimum. Generally, we recommend setting maxGrams to the character count of the longest word in the field that you want to query. If you are unsure, for English language fields, we recommend starting with maxGrams value of 10.

  • Avoid nGram tokenization strategy as, for a given string, Atlas Search creates more tokens for nGram than for edgeGram or rightEdgeGram tokenization.

When indexing a string field as the autocomplete type, we recommend that you index the field as the Atlas Search string type also for the following advantages:

Atlas Search doesn't support indexing more than two billion index objects, where each indexed embedded document counts as a single object. Using the embeddedDocuments field type can result in indexing objects over this limit, which causes an index to transition to a failed state. If your collection has large arrays that might generate two billion objects, you must shard any clusters that contain indexes with the embeddedDocuments type.

If you want to filter and facet your data using the same field, we recommend that you index the field as following Atlas Search types:

For an example of filtering the data by another field for faceting, see the How to Use Facets with Atlas Search tutorial.

Using a multi analyzer to analyze the same field multiple different ways can cause large indexes, especially when analyzing fields with very long values.

You can use the Atlas Search Language Analyzers to index many languages. For the list of languages for which Atlas Search provides built-in analyzers, see the Language Analyzers. For an example, see the How to Run Multilingual Atlas Search Queries tutorial. To index and query languages that are currently not in the list of built-in Language Analyzers, you can create a custom analyzer. For an example, see the Custom Language Analyzer Example.

Suppose you have one document for each language in your collection. Consider the following:

  • You can index the fields separately in the same index using the Language Analyzers. A single index can support multiple languages in the same query.

  • Alternatively, you can create an index per language, which is useful in isolating the different language documents. Note that each index is a change stream cursor and so this might be expensive to maintain.

  • If you have the language documents nested inside a parent document, you can create a single index. However, your index definition payload might be large and your query might be complex.

To learn more about these data models and index definitions, see the MongoDB blog.

Inserts and updates to a synonym source collection are fast only if the synonym source collection is small. For best performance, we recommend batching inserts and updates to synonym source collections.

A synonym mapping definition doesn't require additional disk space aside from the disk space utilized by the synonym collection in the database. However, synonym mappings create artifacts in memory and therefore, for synonym collections with many documents, Atlas Search creates artifacts that occupy more memory.

Creating an Atlas Search index is resource-intensive. The performance of your Atlas cluster may be impacted while the index builds.

Atlas replicates all writes on the collection. This means that for each collection with Atlas Search indexes, the writes are amplified to the amount of Atlas Search indexes defined for that collection.

In some instances, your Atlas Search index must be rebuilt. Rebuilding the Atlas Search index also consumes resources and may affect database performance. Atlas Search automatically rebuilds the index only in the event of:

  • Changes to the index definition

  • Atlas Search version updates that include breaking changes

  • Hardware-related problems such as index corruption

Note

Atlas Search supports no-downtime indexing, which means you can continue to run search queries while Atlas Search rebuilds your index. Atlas Search keeps your old index up-to-date while the new index is being built. We recommend allocating free disk space equal to 125% of the disk space used by your old index for this operation. You can view the amount of disk space currently used by your index in the Search Disk Space Used metric.

If your index rebuild fails due to insufficient disk space, we recommend that you temporarily expand your cluster capacity to meet the increased demand. You can make this change manually as described in Fix Storage Issues, even for clusters with autoscaling enabled.

Once Atlas Search rebuilds the index, the old index is automatically replaced without any further action from your side.

Atlas Search supports eventual consistency and does not provide any stronger consistency guarantees. This means that data inserted into a MongoDB collection and indexed by Atlas Search will not be available immediately for $search queries.

Atlas Search reads data from MongoDB change streams and indexes that data in an asynchronous process. This process is typically very fast, but may sometimes be impacted by replication latency, system resource availability, and index definition complexity.

Mapping explosions occur when Atlas Search indexes a document with arbitrary keys and you have a dynamic mapping. The mongot process might consume increasing amounts of memory and could crash. If you add too many fields to an index, mapping explosions can occur. To address this issue, you can upgrade your cluster or use a static mapping that does not index all fields in your data.

When searching over fields using a wildcard path, design your search to use a tuple-like schema. If you perform a wildcard path search that uses a key-value schema, Atlas Search indexes each key as its own field, which can cause mapping explosions.

Example

An example of a key-value schema is as follows:

ruleBuilder: {
ruleName1: <data>,
ruleName2: <data>,
.....
ruleName1025: <data>
}

An example of the same data restructured to use a tuple-like schema is as follows:

{
ruleBuilder: [
{name: ruleName1, data: <data>},
{name: ruleName2, data: <data>},
...
{name: ruleName1025, data: <data>}
]
}

You can configure fields to store on Atlas Search and improve performance of subsequent aggregation pipeline stages like $sort, $match, $group, and $skip. Use this optimization if your original documents and matched dataset are so large that a full data lookup is inefficient. To learn more about storing specific fields on Atlas Search and returning those stored fields only, see Define Stored Source Fields in Your Atlas Search Index and Return Stored Source Fields.

We recommend storing only the minimum number of fields required for subsequent stages. If necessary, you can use $lookup at the end of the pipeline stage to retrieve entire documents as shown in the Examples. Storing unnecessary fields increases disk utilization and could negatively impact performance during indexing and querying.

Atlas Search is deployed on your Atlas cluster. When a new version of Atlas Search is deployed, your Atlas cluster might experience brief network failures in returning query results. To mitigate issues during deployment and minimize impact to your application, consider the following:

To learn more about the changes in each release, see Atlas Search Changelog.

You can scale up your initial sync and steady state indexing for an Atlas Search index by upgrading your cluster to a higher tier with more cores. Atlas Search uses a percentage of all available cores to run both initial sync and steady state indexing and performance improves as new cores are made available by upgrading your cluster.

If you reconfigure your deployment to use the local NVMe storage type or upscale an NVMe-based cluster, Atlas Search performs an initial sync of all configured Atlas Search indexes after each node completes its underlying configuration or upscale action. If the Atlas Search index initial syncs take longer than the time it took to complete the cluster configuration change, you can't run $search queries until the initial sync completes on all the nodes in your Atlas cluster.

We recommend deploying dedicated Search Nodes to scale your Atlas cluster and $search workloads independently. Dedicated Search Nodes run only the mongot process and therefore improve the availability, performance, and workload balancing of the mongot process.

←  Improve Atlas Search PerformanceAtlas Search Query Performance →