Manage Indexes

View Existing Indexes

Remove Indexes

Modify an Index

Find Inconsistent Indexes across Shards

A sharded collection has an inconsistent index if the collection does not have the exact same indexes (including the index options) on each shard that contains chunks for the collection. Although inconsistent indexes should not occur during normal operations, inconsistent indexes can occur , such as:

• When a user is creating an index with a unique key constraint and one shard contains a chunk with duplicate documents. In such cases, the create index operation may succeed on the shards without duplicates but not on the shard with duplicates.

• When a user is creating an index across the shards in a rolling manner (i.e. manually building the index one by one across the shards) but either fails to build the index for an associated shard or incorrectly builds an index with different specification.

Starting in MongoDB 4.4 (and 4.2.6), the config server primary, by default, checks for index inconsistencies across the shards for sharded collections, and the command serverStatus, when run on the config server primary, returns the field shardedIndexConsistency field to report on the number of sharded collections with index inconsistencies.

If shardedIndexConsistency reports any index inconsistencies, you can run the following pipeline for your sharded collections until you find the inconsistencies.

1. Define the following aggregation pipeline:

   const pipeline = [
       // Get indexes and the shards that they belong to.
       {$indexStats: {}},
       // Attach a list of all shards which reported indexes to each document from $indexStats.
       {$group: {_id: null, indexDoc: {$push: "$$ROOT"}, allShards: {$addToSet: "$shard"}}},
       // Unwind the generated array back into an array of index documents.
       {$unwind: "$indexDoc"},
       // Group by index name.
       {
           $group: {
               "_id": "$indexDoc.name",
               "shards": {$push: "$indexDoc.shard"},
               // Convert each index specification into an array of its properties
               // that can be compared using set operators.
               "specs": {$push: {$objectToArray: {$ifNull: ["$indexDoc.spec", {}]}}},
               "allShards": {$first: "$allShards"}
           }
       },
       // Compute which indexes are not present on all targeted shards and
       // which index specification properties aren't the same across all shards.
       {
           $project: {
               missingFromShards: {$setDifference: ["$allShards", "$shards"]},
               inconsistentProperties: {
                    $setDifference: [
                        {$reduce: {
                            input: "$specs",
                            initialValue: {$arrayElemAt: ["$specs", 0]},
                            in: {$setUnion: ["$$value", "$$this"]}}},
                        {$reduce: {
                            input: "$specs",
                            initialValue: {$arrayElemAt: ["$specs", 0]},
                            in: {$setIntersection: ["$$value", "$$this"]}}}
                    ]
                }
           }
       },
       // Only return output that indicates an index was inconsistent, i.e. either a shard was missing
       // an index or a property on at least one shard was not the same on all others.
       {
           $match: {
               $expr:
                   {$or: [
                       {$gt: [{$size: "$missingFromShards"}, 0]},
                       {$gt: [{$size: "$inconsistentProperties"}, 0]},
                   ]
               }
           }
       },
       // Output relevant fields.
       {$project: {_id: 0, indexName: "$$ROOT._id", inconsistentProperties: 1, missingFromShards: 1}}
   ];

2. Run the aggregation pipeline for the sharded collection to test. For example, to test if the sharded collection test.reviews has inconsistent indexes across its associated shards:

   db.getSiblingDB("test").reviews.aggregate(pipeline)

   If the collection has inconsistent indexes, the aggregation for that collection returns details regarding the inconsistent indexes:

   { "missingFromShards" : [ "shardB" ], "inconsistentProperties" : [ ], "indexName" : "page_1_score_1" }
   { "missingFromShards" : [ ], "inconsistentProperties" : [ { "k" : "expireAfterSeconds", "v" : 60 }, { "k" : "expireAfterSeconds", "v" : 600 } ], "indexName" : "reviewDt_1" }

   The returned documents indicate two inconsistencies for the sharded collection test.reviews:

   1. An index named page_1_score_1 is missing from the collection on shardB.

   2. An index named reviewDt_1 has inconsistent properties across the collection's shards, specifically, the expireAfterSeconds properties differ.

To resolve the inconsistency where an index is missing from the collection on a particular shard(s),

You can either:

• Perform a rolling index build for the collection on the affected shard(s).

  -OR-

• Issue an index build db.collection.createIndex() from a mongos instance. The operation only builds the collection's index on the shard(s) missing the index.

To resolve where the index properties differ across the shards,

Drop the incorrect index from the collection on the affected shard(s) and rebuild the index. To rebuild the index, you can either:

• Perform a rolling index build for the collection on the affected shard.

  -OR-

• Issue an index build db.collection.createIndex() from a mongos instance. The operation only builds the collection's index on the shard(s) missing the index.

Alternatively, if the inconsistency is the expireAfterSeconds property, you can run the collMod command to update the number of seconds instead of dropping and rebuilding the index.