Docs Home → MongoDB Cluster-to-Cluster Sync
oplog Sizing
On this page
The mongosync program uses change streams to synchronize data between source and destination
clusters. mongosync does not access the oplog directly,
but when a change stream returns events from the past, the events must
be within the oplog time range.
mongosync applies operations in the oplog on the source cluster
to the data on the destination cluster. When operations
that mongosync has not applied roll off the oplog
on the source cluster, the sync fails and mongosync exits.
During the initial sync, mongosync may apply operations at a slower
rate due to the load imposed by copying documents concurrently.
After mongosync completes the initial sync, it applies changes
faster and is more likely to maintain a position in the oplog
that is close to the real-time writes occuring on the source cluster.
If you anticipate syncing a large data set, or if you plan to pause
synchronization for an extended period of time, you might exceed the
oplog window. Use the oplogSizeMB setting
to increase the size of the oplog on the source cluster.
Monitor oplog Size Needed for Initial Sync
Determine oplog Window
To get the difference in seconds between the first and last entry
in the oplog run db.getReplicationInfo(). If you
are replicating a sharded cluster, run the command on each shard.
db.getReplicationInfo().timeDiff
The value returned is the minimum oplog window of the
cluster. If there are multiple shards, the smallest number is the
minimum oplog window.
Determine mongosync Replication Lag
To get the lagTimeSeconds value, run the
/progress command.
The lag time is the time in seconds between the
last event applied by mongosync and time of the current
latest event on the source cluster.
It is a measure of how far behind the source cluster mongosync is.
Validate oplog Size
If the lag time approaches the minimum oplog window, make
one of the following changes:
Increase the
oplogwindow. UsereplSetResizeOplogto setminRetentionHoursgreater than the currentoplogwindow.Scale up the
mongosyncinstance. Add CPU or memory to scale up themongosyncnode so that it has a higher copy rate.
Note
The oplog window and rate of change for replication lag may vary during synchronization. Repeat these steps during a migration to monitor the progress.