start

State

To use the start endpoint, mongosync must be in the IDLE state.

Permissions

The user specified in the mongosync connection string must have the required permissions on the source and destination clusters. Refer to User Permissions to ensure that the user has the correct permissions to start the synchronization.

Multiple mongosync Instances

Ensure that you use the configured mongosync user in the connection strings for the cluster0 or cluster1 settings when you start mongosync.

Request Body Parameters

db.adminCommand( { setUserWriteBlockMode: 1, global: false } )

Sharding Parameters

To sync from a replica set to a sharded cluster, set the sharding option to shard collections on the destination cluster.

mongosync throws an error if the sharding option is not set when syncing from a replica set to a sharded cluster. mongosync also throws an error if the sharding option is set with any other configuration.

The sharding option has the following parameters:

Start a Sync Job

The following example starts a sync job between the source cluster cluster0 and the destination cluster cluster1.

Request:

curl localhost:27182/api/v1/start -XPOST \
--data '
   {
      "source": "cluster0",
      "destination": "cluster1"
   } '

Response:

{"success":true}

Start a Reversible Sync Job

The following example starts a sync job between the source cluster cluster0 and the destination cluster cluster1.

The reversible and enableUserWriteBlocking fields allow the sync to be reversed. To reverse the sync direction, see: reverse.

Request:

curl localhost:27182/api/v1/start -XPOST \
--data '
   {
      "source": "cluster0",
      "destination": "cluster1",
      "reversible": true,
      "enableUserWriteBlocking": "sourceAndDestination"
   } '

Response:

{"success":true}

Start a Filtered Sync Job

The following example starts a sync job between the source cluster cluster0 and the destination cluster cluster1.

cluster0 contains the sales, marketing, and engineering databases.

The sales database contains the EMEA, APAC, and AMER collections.

The includeNamespaces array in this example defines a filter on two of the databases, sales and marketing.

The sales database also filters on the EMEA and APAC collections.

"includeNamespaces" : [
      {
          "database" : "sales",
          "collections": [ "EMEA", "APAC" ]
      },
      {
          "database" : "marketing"
      }
   ]

After you call the /start API with this filter in place, mongosync:

• Syncs all of the collections in the marketing database

• Filters out the engineering database

• Syncs the EMEA and APAC collections from the sales database

• Filters out the AMER collection

The includeNamespaces option creates a filter. To filter the sync, see: Filtered Sync

Request:

curl -X POST "http://localhost:27182/api/v1/start" --data '
   {
      "source": "cluster0",
      "destination": "cluster1",
      "includeNamespaces": [
         {
             "database": "sales",
             "collectionsRegex": {
                "pattern": "^accounts_.+$",
                "options": "i"
             }
         }, {
            "database": "marketing"
         }
      ]
   } '

Response:

{"success":true}

Start Sync from Replica Set to Sharded Cluster

The following example starts a sync job between the source replica set cluster0 and the destination sharded cluster cluster1. The key array in this example defines the shard key {"location": 1, "region": 1 }.

Request:

curl localhost:27182/api/v1/start -XPOST \
--data '
   {
      "source": "cluster0",
      "destination": "cluster1",
      "sharding": {
         "createSupportingIndexes": true,
         "shardingEntries": [
            {
                "database": "accounts",
                "collection": "us_east",
                "shardCollection": {
                   "key": [
                      { "location": 1 },
                      { "region": 1 }
                   ]
                }
            }
         ]
      }
   } '

Response:

{"success":true}

Start with the Verifier Disabled

Starting in 1.9, the embedded verifier runs by default when you start a migration. If you need to disable it, set verification.enabled to false.

Request:

curl localhost:27182/api/v1/start -XPOST \
--data '
   {
      "source": "cluster0",
      "destination": "cluster1",
      "verification": {
         "enabled": false
      }
   } '

Response:

{"success":true}

You should verify that a migration was successful before transferring your application load to the destination cluster. If you need to disable the verifier for any reason, use an alternative method to verify sync.

Embedded Verifier

Starting in 1.9, mongosync provides an embedded verifier to determine whether data transfer from the source cluster to the destination was successful.

When enabled, the verifier performs a series of verification checks on the destination cluster. If any of these checks return an error, the verifier fails the migration. If all checks succeed, mongosync transitions to the COMMITTED state.

To disable the verifier, see Start with the Verifier Disabled.

The /start endpoint returns an error if you enable verification checks that are not supported by the source or destination cluster or if there is insufficient memory.

State

If the start request is successful, mongosync enters the RUNNING state.

Shard Replica Sets

Sync from a replica set to a sharded cluster requires the sharding option. This option configures how mongosync shards collections.

The sharding.shardingEntries array specifies the collections to shard. Collections that are not listed in this array replicate as unsharded.

For more information, see Sharded Cluster Behavior.

Supporting Indexes

mongosync syncs indexes from the source cluster to the destination cluster. When syncing from a replica set to a sharded cluster, mongosync may require an additional index to support the shard key, which may not exist on the source cluster.

mongosync can create supporting indexes for sharded collections during sync. This is done by setting the sharding.createSupportingIndexes option.

When sharding.createSupportingIndexes is false (the default):

• Each shard key you provide for the sharding.shardingEntries option must have an existing index on the source cluster.

• One of the indexes used for the shard key must have simple collation if the collection uses any other collation.

• To use a unique index in the shard key, you must specify its uniqueness when you create the index on the source cluster.

• Unique indexes on the source cluster that are incompatible with the requested shard key on the destination cluster, such as a unique index on the source that does not contain the requested shard key as a prefix on the destination, can cause mongosync to fail.

When sharding.createSupportingIndexes is true:

• If the supporting indexes exist on the source cluster, mongosync syncs the indexes to the destination cluster and uses them as shard keys.

• If the supporting indexes don't exist, mongosync creates them on the destination cluster.

The sharding.createSupportingIndexes option affects all sharded collections.

Rename During Sync

Collections listed in the sharding.shardingEntries array when synced from a replica set to a sharded cluster become sharded collections on the destination cluster.

Renaming a collection (such as with the renameCollection command) on the source cluster after calling start but before mongosync begins to copy the collection can block the collection from sharding on the destination.

To check whether it is safe to rename collections, call the progress endpoint and check the value of the collectionCopy.estimatedCopiedBytes field in the return document.

• A value of 0 indicates that mongosync has not started to copy the collection.

  Renaming a collection at this point may result in an unsharded collection on the destination cluster, as the transition to copying can happen before the rename takes effect on the source.

• A value greater than 0 indicates that mongosync has started the copy. Renaming the collection from this point on does not block its sharding on the destination cluster, even in the event of a crash.

Required Indexes

When you call /start with the buildIndexes option set to never, mongosync skips building unnecessary indexes.

Indexes that are always built include:

• mongosync builds an index on the _id field of every collection it copies.

• mongosync builds dummy indexes for each sharded collection that does not have an index to support the shard key on the destination cluster. When buildIndexes is set to never, mongosync retains this index after commit.

Endpoint Protection

mongosync does not protect the start endpoint. However, by default the API binds to localhost only and does not accept calls from other sources. Additionally, the start call does not expose connection credentials or user data.