Mongosync utility oplog time resume is not possible

hello guys hope you are doing well , iam using mongosync utility to sync two cluster which i achieved successfully ,but i want this sync to happen everytime i have installed utility in ec2 ubuntu instance how do i run the commands all the time and since today iam not able to intiate the syncing
iam getting the following error
“error”:"(ChangeStreamHistoryLost) PlanExecutor error during aggregation :: caused by :: Resume of change stream was not possible, as the resume point may no longer be in the oplog.",“time”:“2022-12-15T11:31:02.443635973+03:00”,“message”:“Error during replication”}

would be very happy if someone could share some suggestion and resolution, thanks

Hi @mohamed_aslam

The error indicates the sync is attempting to resume an existing sync. To do that the oplog must be large enough to cope with the time the sync is not running.

You can use replSetResizeOplog to resize the oplog.

my source cluster has oplog window of 6 hours and destination 24hrs which is default, how to do i keep the sync 24/7 without interruption

any suggestion,please i would be glad becuase we want to implement the syncing for reporting and analytic purpose ,please any help would be very highly appreciated and iam struck with this error

You need to make sure that the source oplog window is large enough to cover the initial data copy and whatever pause/resume intervals you have. If you check the mongosync logs, you will see when it started the data copy and when it finished - most likely it took longer than 6 hours. In general, your oplog window should be larger than 6 hours (best practice is 24-48 hours)

Facing same error for mongosync on one of the sharded replicaset cluster as below -
“(ChangeStreamHistoryLost) Error on remote shard cent-mon-dt-ma6:27017 :: caused by :: Executor error during getMore :: caused by :: Resume of change stream was not possible, as the resume point may no longer be in the oplog.”,“message”:“Error during replication”}"

Also, I am seeing there is no lag in replicaset members; sharing screenshot below -
image

Note : - mongorestore done on this sharded replicaset cluseter and then this issue occurs. Also, I noticed that after some time node recovered itselt and same error comes on another nodes in the sharded replicaset cluster for mongosync.

Kindly suggest some solutions on this… as /data size is 500GB oplog is taken enough as around 25GB.

Looking for solution on this.


Thanks,
Ajay Tumsare

Sharing screenshot from Primary for above issue and seems there is no difference in oplog size too -

Another screenshot for error logs for mongosync -

{“time”:“2024-01-11T14:50:22.032011+05:30”,“level”:“fatal”,“serverID”:“3b199ea3”,“mongosyncID”:“coordinator”,“stack”:[{“func”:“(*ChangeStreamReader).run.func1”,“line”:“291”,“source”:“/data/mci/104f37d051a93bddf6d52dbbf2ffab39/src/github.com/10gen/mongosync/internal/mongosync/cea/change_stream_reader.go”},{“func”:“Retryer.RunForTransientErrorsOnly.func1”,“line”:“67”,“source”:“/data/mci/104f37d051a93bddf6d52dbbf2ffab39/src/github.com/10gen/mongosync/internal/mongosync/retry/retry.go”},{“func”:“(*looper).loop.func1”,“line”:“238”,“source”:“/data/mci/104f37d051a93bddf6d52dbbf2ffab39/src/github.com/10gen/mongosync/internal/mongosync/retry/retry.go”},{“func”:“RunAndDetect”,“line”:“37”,“source”:“/data/mci/104f37d051a93bddf6d52dbbf2ffab39/src/github.com/10gen/mongosync/internal/mongosync/util/slowness/slowness.go”},{“func”:“(*looper).loop”,“line”:“233”,“source”:“/data/mci/104f37d051a93bddf6d52dbbf2ffab39/src/github.com/10gen/mongosync/internal/mongosync/retry/retry.go”},{“func”:“Retryer.runRetryLoop”,“line”:“130”,“source”:“/data/mci/104f37d051a93bddf6d52dbbf2ffab39/src/github.com/10gen/mongosync/internal/mongosync/retry/retry.go”},{“func”:“Retryer.RunForTransientErrorsOnly”,“line”:“69”,“source”:“/data/mci/104f37d051a93bddf6d52dbbf2ffab39/src/github.com/10gen/mongosync/internal/mongosync/retry/retry.go”},{“func”:“(*ChangeStreamReader).run”,“line”:“234”,“source”:“/data/mci/104f37d051a93bddf6d52dbbf2ffab39/src/github.com/10gen/mongosync/internal/mongosync/cea/change_stream_reader.go”},{“func”:“(*ChangeStreamReader).Run”,“line”:“151”,“source”:“/data/mci/104f37d051a93bddf6d52dbbf2ffab39/src/github.com/10gen/mongosync/internal/mongosync/cea/change_stream_reader.go”},{“func”:“(*Mongosync).startCEAComponents.func1”,“line”:“165”,“source”:“/data/mci/104f37d051a93bddf6d52dbbf2ffab39/src/github.com/10gen/mongosync/internal/mongosync/change_event_application.go”},{“func”:“(*MongoErrGroup).NewRoutine.func2”,“line”:“49”,“source”:“/data/mci/104f37d051a93bddf6d52dbbf2ffab39/src/github.com/10gen/mongosync/internal/mongosync/util/routines.go”},{“func”:“(*Group).Go.func1”,“line”:“75”,“source”:“/data/mci/104f37d051a93bddf6d52dbbf2ffab39/src/github.com/10gen/mongosync/vendor/golang.org/x/sync/errgroup/errgroup.go”},{“func”:“goexit”,“line”:“1598”,“source”:“/opt/golang/go1.20/src/runtime/asm_amd64.s”}],“error”:“failed to iterate change stream: Change Stream Reader (CRUD) could not retrieve the next change event (last resume token is {"_data": "826585257600005E0A2B042C0100296E5A10048E9BFDBD5705491BB0D8EAAD1ED87755463C6F7065726174696F6E54797065003C696E736572740046646F63756D656E744B657900463C5F6964003C303038363734656566356461633363363662666335613032313336326336636600000004"}): (ChangeStreamHistoryLost) Error on remote shard cent-mon-dt-ma6:27017 :: caused by :: Executor error during getMore :: caused by :: Resume of change stream was not possible, as the resume point may no longer be in the oplog.”,“message”:“Error during replication”}

==========================================================