mongosync usage and instructions on how
to upgrade your version of mongosync.This page provides answers to some frequently asked questions we have encountered. If you have additional questions please contact MongoDB Support.
Can I perform reads or writes to my destination cluster while mongosync is syncing?
You can perform reads during synchronization at any time. However, the data that you read is eventually consistent. For consistent reads, wait for the migration to commit. To learn more, see Considerations for Continuous Sync.
If you write to a synced namespace before issuing a commit and while canWrite is false, the behavior is
undefined. To ensure that you don't write to any synced namespaces, enable
write blocking.
Note
Index builds on the destination cluster are treated as writes
while mongosync is syncing.
To check the value of canWrite, call the progress
API endpoint.
Can mongosync run on its own hardware?
Yes, mongosync can run on its own hardware. mongosync does not
have to run on the servers that host your MongoDB instances. When
mongosync runs on its own hardware, it can use an operating system
(OS) that is different than the OS on the source or destination
clusters.
Should I increase the size of the oplog in the source cluster?
mongosync applies operations in the oplog on the source cluster
to the data on the destination cluster. When operations
that mongosync has not applied roll off the oplog
on the source cluster, the sync fails and mongosync exits.
Note
mongosync does not replicate applyOps operations made on
the source cluster during sync to the destination cluster.
During the initial sync, mongosync may apply operations at a slower
rate due to copying documents concurrently.
After the initial sync, mongosync applies changes
faster and is more likely to maintain a position in the oplog
that is close to the real-time writes occurring on the source cluster.
If you anticipate syncing a large data set, or if you plan to pause
synchronization for an extended period of time, you might exceed the
oplog window. Use the oplogSizeMB setting
to increase the size of the oplog on the source cluster.
Which connection string options does mongosync allow?
mongosync requires readConcern: "majority"
and writeConcern: "majority".
If the readConcern is not majority, mongosync returns an
error:
Invalid URI option, read concern must be majority
If the writeConcern is not majority, mongosync returns an
error:
Invalid URI option, write concern must be majority
mongosync accepts all other connection string options.
Which security and authentication options are supported?
mongosync uses a standard MongoDB connection string to connect to the source and destination clusters.
LDAP and X509 are supported. For available authentication options, see Authentication on Self-Managed Deployments.
Does mongosync automatically restart on error?
mongosync does not automatically restart on error. However, you
can write a script or use your operating system's process managers,
systemd for example, to restart the mongosync process.
The mongosync binary is stateless. The metadata for restarting is
stored on the destination cluster.
A mongosync operation can be resumed if mongosync becomes
unavailable during synchronization. When mongosync becomes
available again, restart the mongosync process with the same
parameters. mongosync resumes the operation from where it stopped
when mongosync became unavailable.
Note
Starting in mongosync 1.7.3, mongosync can take at least two minutes
to respond when you resume or restart a sync operation. During this time,
any calls to the progress endpoint might fail. If a
progress call fails, it is safe to retry.
Can the source or destination be a replica set with arbiters?
Yes, the replica set can have arbiters. The source replica set must have more than 2 non-arbiter nodes and you must sync from a non-arbiter node. Use the source cluster's connection string to specify a read preference for a non-arbiter, data-bearing node.
What if I see a Slow Operation Warning?
Slow operation warnings can occur during the initial sync or the application of a change event when there is a slow read operation on the source cluster or a slow write operation on the destination cluster. The warning may indicate network congestion or resource strain on the source or destination cluster.
While these warnings do not indicate failures in themselves, slow operations
can cause operation timeout errors in mongosync and migration failures.
If you see slow operation warnings, check CPU, memory, and network usage on the source and destination clusters. If the clusters are underprovisioned for your needs, consider upgrading the cluster hardware.
Should I stop a migration if the logs contain the word "error" or "failure"?
No, logs that contain the word "error" or "failure" show non-fatal
errors and do not signal that you need to stop mongosync early.
These logs do not indicate that mongosync is failing or corrupting
data. If a fatal error occurs, mongosync stops the sync and writes a
fatal log entry.
What if I see a lot of duplicate key errors in the logs?
Duplicate key errors are a normal part of the sync process. These errors can occur if:
You insert a document on the source cluster after
mongosyncstarts.mongosyncmay directly copy the document and redundantly apply the insert change event for the document later.You stop and resume
mongosync. This can lead to duplicate insertions whenmongosyncrestarts.mongosyncencounters a transient error and reattempts an insertion that may have already succeeded.
What should I do if mongosync returns a fatal error?
A fatal error indicates a problem that must be corrected and requires the
migration to be restarted. After addressing the error, delete all migrated
data on the destination cluster, including the mongosync_reserved_for_internal_use
database. Then, relaunch mongosync and start a new migration.
Does mongosync Support TTL Indexes?
Cluster-to-Cluster Sync supports syncing TTL Indexes from the source to the destination cluster.