Docs Home → MongoDB Cluster-to-Cluster Sync
oplog
Sizing
On this page
The mongosync program uses change streams to synchronize data between source and destination
clusters. mongosync
does not access the oplog directly,
but when a change stream returns events from the past, the events must
be within the oplog time range.
The oplog in the source cluster must be large enough to track events that happen during the time it takes to complete the initial sync to the destination cluster.
If you anticipate syncing a large data set, or if you plan to pause
synchronization for an extended period of time, increase the size of the
replica set oplog
in the source cluster.
Estimate oplog
Size Needed for Initial Sync
Determine the oplog
Time Span
To get the difference in seconds between the first and last entry
in the oplog
run db.getReplicationInfo()
. If you
are replicating a sharded cluster, run the command on each shard.
db.getReplicationInfo().timeDiff
The value returned is the minimum oplog
window of the
cluster. If there are multiple shards, the smallest number is the
minimum oplog
window.
Estimate Copy Rate During Synching
To gather performance data while synching, start the sync process and monitor how fast data is transferred between clusters.
To start syncing, run the /start command.
To get the copy_rate
:
run the /progress command to get
estimatedCopiedBytes_time01
wait a second or two
run the /progress command to get
estimatedCopiedBytes_time02
The copy_rate
is:
copy_rate = ( estimatedCopiedBytes_time02 - estimatedCopiedBytes_01) / time_between_requests
Validate oplog
Size
If the estimated time is larger than the minimum oplog window you must cancel synchronization. Before restarting, make one of the following changes:
Increase the oplog window. Use
replSetResizeOplog
to setminRetentionHours
greater than the estimated copy time.Scale up the
mongosync
instance. Add cpu or memory to scale up themongosync
node so that it has a higher copy rate.
Note
The copy rate may vary during synchronization. To monitor progress, repeat the steps to estimate the copy rate and verify that the copy rate stays about the same.