Use mongosync on Sharded Clusters

Configure a Single mongosync Instance

To configure a single mongosync, follow the connection instructions for your cluster architecture to connect to the mongos instance in your cluster.

When you connect a single mongosync to a sharded cluster do not use the replicaSet option or the id option.

The rest of this page addresses cluster to cluster synchronization using multiple monogosync instances.

Configure Multiple mongosync Instances

To configure multiple mongosync instances:

1. Verify cluster configuration

2. Determine the shard IDs

3. Connect the instances

sh.stopBalancer()

db.adminCommand( { listShards: 1 } )

  shards: [
    {
      _id: 'shard01',
      host: 'shard01/localhost:27501,localhost:27502,localhost:27503',
      state: 1,
      topologyTime: Timestamp({ t: 1656612236, i: 2 })
    },
    {
      _id: 'shard02',
      host: 'shard02/localhost:27504,localhost:27505,localhost:27506',
      state: 1,
      topologyTime: Timestamp({ t: 1656612240, i: 4 })
    }
  ]

mongosync \
  --cluster0 "mongodb://user:password@cluster0host:27500" \
  --cluster1 "mongodb://user:password@cluster1host:27500" \
  --id shard01 --port 27601

 mongosync \
   --cluster0 "mongodb://user:password@cluster0host:27500" \
   --cluster1 "mongodb://user:password@cluster1host:27500" \
   --id shard02 --port 27602

Start Multiple mongosync Instances

Use curl or another HTTP client to issue the start command to each of the mongosync instances.

curl mongosync01Host:27601/api/v1/start -XPOST --data \
     '{ "source": "cluster0", "destination": "cluster1", \
        "reversible": false, "enableUserWriteBlocking": false }'
curl mongosync02Host:27602/api/v1/start -XPOST --data \
     '{ "source": "cluster0", "destination": "cluster1", \
        "reversible": false, "enableUserWriteBlocking": false }'

The start command options must be the same for all of the mongosync instances.

Check Progress

The mongosync instances do not aggregate progress information across shards. To review synchronization progress for a particular shard, use curl or another HTTP client to issue the progress command to the mongosync instance syncing that shard.

curl mongosync02Host:27602/api/v1/progress -XGET

This command checks the progress of the mongosync instance that is running on mongosync02Host and using port 27602 for synchronization. To check progress on other shards, update the host and port number then repeat the API call to each mongosync instance.

Pause a mongosync Instance

The pause command will temporarily halt the synchronization process on a single shard. It does not pause any other mongosync instances that may be running. Use curl or another HTTP client to issue the pause command to a mongosync instance.

curl mongosync01Host:27601/api/v1/pause -XPOST --data '{}'

This command pauses the mongosync instance that is running on mongosync01Host and using port 27601 for synchronization. To pause synchronization on other shards, update the host and port number then repeat the API call to each mongosync instance.

Resume Synchronization

If one or more mongosync instances are paused, you can use the resume command to resume syncing. Run a separate resume command against each paused mongosync instance to continue syncing.

Use curl or another HTTP client to issue the resume command to each mongosync instance.

curl mongosync01Host:27601/api/v1/resume -XPOST --data '{}'

This command resumes synchronization on the mongosync instance that is running on mongosync01Host and using port 27601. To resume synchronization on other shards, update the host and port number then repeat the API call to each mongosync instance.

Commit Synchronization From Multiple mongosync Instances

When you want to complete synchronization, issue the progress command and check the values for canCommit and lagTimeSeconds.

To minimize write blocking on the source cluster, you should only run the commit command when the lagTimeSeconds value is small enough for your application.

If the lagTimeSeconds value is small enough, and canCommit is true, issue the commit command to commit synchronization. Repeat the process on all of the mongosync instances.

The commit operation is blocking. The commit command will not return until commit has been called on every mongosync instance.

// Check progress
curl mongosync01Host:27601/api/v1/progress -XGET
// Commit
curl mongosync01Host:27601/api/v1/commit -XPOST --data '{}'

These commands only check progress and commit synchronization for the mongosync instance that is running on mongosync01Host and using port 27601. To synchronize all of the shards, make additional calls to progress and commit on any other mongosync instances that may be running.

Once this is done, use sh.startBalancer() to restart the balancer on the destination cluster:

sh.startBalancer()

Reverse the Synchronization Direction

To reverse synchronization so that the original destination cluster acts as the source cluster:

• If you have not already done so, issue the commit command to each mongosync instance and wait until all of the commits to finish.

• Issue the reverse command to each mongosync instance.

The reverse operation is blocking. The reverse command will not return until reverse has been called on every mongosync instance.

curl mongosync01Host:27601/api/v1/reverse -XPOST --data '{}'

This command reverses synchronization on the mongosync instance that is running on mongosync01Host and using port 27601. Make additional calls to reverse on any other mongosync instances that may be running.