Docs Menu
Docs Home
/
MongoDB Cluster-to-Cluster Sync
/

mongosync

On this page

  • Description
  • Options
  • Global Options
  • Behavior
  • Cluster Independence
  • Configuration File
  • Consistency
  • Profiling
  • Sharded Clusters
  • Data Definition Language (DDL) Operations
  • Views
  • Write Blocking
  • Commit
  • Read and Write Concern
  • Read Preference
  • Capped Collections
  • System Collections
  • Resilience
  • UUIDs
  • Sorting
  • Multiple Clusters
  • Examples

The mongosync binary is the primary process used in Cluster-to-Cluster Sync. mongosync migrates data from one cluster to another and can keep the clusters in continuous sync.

This document provides a complete overview of the mongosync command line options.

Note

  • The minimum supported server versions of MongoDB are 6.0.13 and 7.0.8.

  • mongosync supports replica sets and sharded clusters.

  • Standalone MongoDB instances are not supported. Convert the standalone instance to a replica set before using Cluster-to-Cluster Sync.

  • mongosync does not support Atlas shared clusters or serverless instances.

  • For a full list of limitations, see Cluster-to-Cluster Sync Limitations.

--cluster0 <URI>

Sets the connection URI for the first cluster. The first cluster can serve as either the source or the destination in the sync process. Designate the source and destination clusters in the call to the start API endpoint.

For more information on connecting mongosync, see Connections.

To set the --cluster0 option from a configuration file, see the cluster0 setting.

--cluster1 <URI>

Sets the connection URI for the second cluster. The second cluster can serve as either the source or the destination in the sync process. Designate the source and destination clusters in the call to the start API endpoint.

For more information on connecting mongosync, see Connections.

To set the --cluster1 option from a configuration file, see the cluster1 setting.

--config <filename>

Sets the path to the configuration file.

For more information, see Cluster Independence.

--disableTelemetry

New in version 1.4.0.

Disables the collection of telemetry data for mongosync.

By default, mongosync collects anonymous, aggregated usage data to improve MongoDB products. When you run mongosync you can disable collection of this telemetry data.

To set the --disableTelemetry option from a configuration file, see the disableTelemetry setting.

For more information, see User Data Collection.

--help, -h

Prints usage information to stdout.

--id <ID>

Sets an identifier for the mongosync instance.

Use this option when running multiple instances of mongosync on a sharded cluster, to synchronize the shards individually.

The identifier value for this option must correspond to the shard ID of the shard it syncs. To find the shard ID, use the listShards command.

To set the --id option from a configuration file, see the id setting.

--loadLevel <level>

Type: integer

Default: 3

Sets the cluster workload level for syncing data between the source and destination clusters:

  • 4, the highest setting, maximizes cluster workload and syncs data the fastest.

  • 1, the lowest setting, minimizes cluster workload and syncs data the slowest.

To set the --loadLevel option from a configuration file, see the loadLevel setting.

Warning

Setting loadLevel higher than the default of 3 may negatively impact the destination cluster performance.

--logPath <DIR>

Sets the path to the log directory. Cluster-to-Cluster Sync writes logs to files in this directory.

For more information, see Logging.

To set the --logPath option from a configuration file, see the logPath setting.

--port

Default: 27182

Sets the port used by the HTTP server for the Cluster-to-Cluster Sync HTTP API.

To set the --port option from a configuration file, see the port setting.

--verbosity <level>

Default: DEBUG

Sets the verbosity level to use in log messages. Cluster-to-Cluster Sync logs all messages at the specified level and any messages at lower levels.

The --verbosity option supports the following values:

  • TRACE

  • DEBUG

  • INFO

  • WARN

  • ERROR

  • FATAL

  • PANIC

To set the --verbosity option from a configuration file, see the verbosity setting.

--version, -v

Prints mongosync version information to stdout.

mongosync syncs collection data between a source cluster and destination cluster. mongosync does not synchronize users or roles. As a result, you can create users with different access permissions on each cluster.

Options for mongosync can be set in a YAML configuration file. Use the --config option. For example:

$ mongosync --config /etc/mongosync.conf

For information on available settings, see Configuration.

mongosync supports eventual consistency on the destination cluster. Read Consistency is not guaranteed on the destination cluster, the source and destination clusters may differ at a given point in time. However, if new writes are halted on the source cluster, the source and destination clusters will eventually hold the same data.

Individual writes may be reordered or combined as they are relayed from source to destination. For a given document, the total number of writes may differ between source and destination.

Transactions may appear non-atomically on the destination cluster. Retryable writes may not be retryable on the destination cluster.

If profiling is enabled on a source database, MongoDB creates a special collection named <db>.system.profile. After synchronization is complete, Cluster-to-Cluster Sync will not drop the <db>.system.profile collection from the destination even if the source database is dropped at a later time. The <db>.system.profile collection will not change the accuracy of user data on the destination.

Cluster-to-Cluster Sync supports replication between sharded clusters. Individual shards are replicated in parallel from the source cluster to the destination cluster, however a chunk migration or similar source update could move documents to a new source shard during replication.

Even if documents move between source shards during replication, Cluster-to-Cluster Sync maintains the eventual consistency guarantee on the destination cluster.

Using DDL operations (operations that act on collections or databases such as db.createCollection() and db.dropDatabase()) during sync increase the risk of migration failure and may negatively impact mongosync performance. For best performance, refrain from performing DDL operations on the source cluster while the sync is in progress.

For more information on DDL operations, see Pending DDL Operations and Transactions.

If a database with views is dropped on the source, the destination may show an empty system.views collection in that database. The empty system.views collection will not change the accuracy of user data on the destination.

mongosync does not enable write-blocking by default. If you enable write-blocking, mongosync blocks writes:

  • On the destination cluster during sync.

  • On the source cluster when commit is received.

To enable write-blocking, use the start API to set enableUserWriteBlocking to true. You cannot enable write-blocking after the sync starts.

You must enable write-blocking when you start mongosync if you want to use reverse synchronization later.

To set enableUserWriteBlocking, the mongosync user must have a role that includes the setUserWriteBlockMode and bypassWriteBlockingMode ActionTypes.

Note

When using enableUserWriteBlocking, writes are only blocked for users that do not have the bypassWriteBlockingMode ActionType. Users who have this ActionType are able to perform writes.

Read operations on the source cluster are always permitted.

When the /progress endpoint reports canWrite is true, the data on the source and destination clusters is consistent.

To see what state mongosync is in, call the /progress API endpoint. The /progress output includes a boolean value, canWrite.

  • When canWrite is true, it is safe to write to the destination cluster.

  • When canWrite is false, do not write to the destination cluster.

You can safely write to the source cluster while mongosync is syncing. Do not write to the destination cluster unless canWrite is true.

To stop syncing, use the commit command on the destination cluster to convert indexes and finalize the changes. If you enable write-blocking:

  • commit blocks writes on the source cluster.

  • commit blocks writes on the destination cluster until mongosync begins index validation.

To monitor the commit process, use the progress command:

  • The destination cluster is writable when canWrite is true.

  • Index conversion is complete when mongosync enters the COMMITTED state.

By default, mongosync sets the read concern level to "majority" for reads on the source cluster. For writes on the destination cluster, mongosync sets the write concern level to "majority" with j: true.

For more information on read and write concern configuration and behavior, see Read Concern and Write Concern.

mongosync requires the primary read preference when connecting to the source and destination clusters. For more information, see Read Preference Options.

Starting in 1.3.0, Cluster-to-Cluster Sync supports capped collections with some limitations.

Capped collections on the source cluster work normally during sync.

Capped collections on the destination cluster have temporary changes during sync:

  • There is no maximum number of documents.

  • The maximum collection size is 1PB.

mongosync restores the original values for maximum number of documents and maximum document size during commit.

Cluster-to-Cluster Sync does not replicate system collections to the destination cluster.

If you issue a dropDatabase command on the source cluster, this change is not directly applied on the destination cluster. Instead, Cluster-to-Cluster Sync drops user collections and views in the database on the destination cluster, but it does not drop system collections on that database.

For example, on the destination cluster:

  • The drop operation does not affect a user-created system.js collection.

  • If you enable profiling, the system.profile collection remains.

  • If you create views on the source cluster and then drop the database, replicating the drop removes the views, but leaves an empty system.views collection.

In these cases, the replication of dropDatabase removes all user-created collections from the database, but leaves its system collections on the destination cluster.

mongosync is resilient and able to handle non-fatal errors. Logs that contain the word "error" or "failure" do not indicate that mongosync is failing or corrupting data. For example, if a network error occurs, the mongosync log may contain the word "error' but mongosync is still able to complete the sync. In the case that a sync does not complete, mongosync writes a fatal log entry.

mongosync creates collections with new UUIDs on the destination cluster. There is no relationship between UUIDs on the source cluster and the destination cluster. If applications contain hard-coded UUIDs (which MongoDB does not recommend), you may need to update those applications before they work properly with the migrated cluster.

mongosync inserts documents on the destination cluster in an undefined order which does not preserve natural sort order from the source cluster. If applications depend on document order but don't have a defined sort method, you may need to update those applications to specify the expected sort order before the applications work properly with the migrated cluster.

To sync a source cluster to multiple destination clusters, use one mongosync instance for each destination cluster. For more information, see Multiple Clusters Limitations.

  1. Deploy a source and a destination cluster.

  2. Initialize Cluster-to-Cluster Sync:

    mongosync \
    --cluster0 "mongodb://192.0.2.10:27017,192.0.2.11:27017,192.0.2.12:27017" \
    --cluster1 "mongodb://192.0.2.20:27017,192.0.2.21:27017,192.0.2.22:27017"

    Use the appropriate connection strings for the --cluster0 and --cluster1 options so that they can connect to your replica sets.

  3. To start the synchronization process, use an HTTP client such as cURL to send the start command to mongosync:

    curl localhost:27182/api/v1/start -X POST \
    --data '{ "source": "cluster0", "destination": "cluster1" }'

    Example Output:

    { "success": true }

Back

Reference

Next

Configuration