Reshard a Collection

About this Task

• Only one collection can be resharded at a time.

• writeConcernMajorityJournalDefault must be true.

• To reshard a collection that has a uniqueness constraint, the new shard key must satisfy the unique index requirements for any existing unique indexes.

• The following commands and corresponding shell methods are not supported on the collection that is being resharded while the resharding operation is in progress:

  • collMod

  • convertToCapped

  • createIndexes

  • createIndex()

  • drop

  • drop()

  • dropIndexes

  • dropIndex()

  • renameCollection

  • renameCollection()

• If you run one of the following operations, the operation waits until resharding completes before executing:

  • addShard

  • removeShard

  • dropDatabase on the database hosting the collection undergoing resharding

• If the collection you're resharding uses Atlas Search, the search index becomes unavailable when the resharding operation completes. You need to manually rebuild the search index once the resharding operation completes.

Before you Begin

Before you reshard your collection, ensure that you meet the following requirements:

• Your application can tolerate a period of two seconds where the affected collection blocks writes. During the time period where writes are blocked, your application experiences an increase in latency.

  If your workload cannot tolerate this requirement, consider refining your shard key instead.

• Your database meets these resource requirements:

  • Ensure that the available storage space on each recipient shard is at least twice the storage size of the collection that you want to reshard plus its total index size, divided by the number of shards:

    ( ( collection_storage_size + index_size ) * 2 ) / shard_count = storage_req

    For example, consider a collection with a storage size of 2 TB data and a 400 GB index. To distribute it across four shards you'd need:

    ( ( 2 TB collection + 0.4 TB index ) * 2 ) / 4 shards = 1.2 TB storage

    To reshard this collection, each shard requires 1.2 TB of available storage.

    On MongoDB Atlas, you may need to upgrade to the next tier of storage for the resharding operation. You can downgrade once the operation completes.

  • Ensure that your I/O capacity is below 50%.

  • Ensure that your CPU load is below 80%.

  Important

  These requirements are not enforced by the database. A failure to allocate enough resources can result in:

  • the database running out of space and shutting down

  • decreased performance

  • the operation taking longer than expected

  If your application has time periods with less traffic, perform this operation on the collection during that time if possible.

• You do not need to create an index on the new shard key before resharding. The resharding operation builds the required indexes automatically during the index phase.

• No index builds are in progress. To check for running index builds, use $currentOp:

  db.getSiblingDB("admin").aggregate( [
     { $currentOp : { idleConnections: true } },
     { $match: {
           $or: [
               { "op": "command", "command.createIndexes": { $exists: true } },
               { "op": "none", "msg": /^Index Build/ }
           ]
        }
     }
  ] )

  In the result document, if the inprog field value is an empty array, there are no index builds in progress:

  {
     inprog: [],
     ok: 1,
     '$clusterTime': { ... },
     operationTime: <timestamp>
  }

Steps

In a collection resharding operation, a shard can be a:

• donor, which currently stores chunks for the sharded collection.

• recipient, which stores new chunks for the sharded collection based on the shard keys and zones.

A shard can be donor and a recipient at the same time.

The config server primary is always the resharding coordinator and starts each phase of the resharding operation.

db.adminCommand({
  reshardCollection: "<database>.<collection>",
  key: <shardkey>
})

db.adminCommand({
  reshardCollection: "<database>.<collection>",
  key: <shardkey>,
  forceRedistribution: true
})

db.getSiblingDB("admin").aggregate([
  { $currentOp: { allUsers: true, localOps: false } },
  {
    $match: {
      type: "op",
      "originatingCommand.reshardCollection": "<database>.<collection>"
    }
  }
])

[
  {
    shard: '<shard>',
    type: 'op',
    desc: 'ReshardingRecipientService | ReshardingDonorService | ReshardingCoordinatorService <reshardingUUID>',
    op: 'command',
    ns: '<database>.<collection>',
    originatingCommand: {
      reshardCollection: '<database>.<collection>',
      key: <shardkey>,
      unique: <boolean>,
      collation: { locale: 'simple' }
    },
    totalOperationTimeElapsedSecs: <number>,
    remainingOperationTimeEstimatedSecs: <number>,
    ...
  },
  ...
]

Behavior

Error Case