Both secondary nodes are stale in mongodb replicaset

Hi experts,
we are on mongodb version: 2.6.12
we have a mongodb replicaset with 3 nodes. 1 primary 2 secondary instances.

Primary is working and both secondary nodes are stale in mongodb replicaset.
Secondary nodes are stuck in recovering state and many days behind primary.
dbpath directory has 5TB data on each secondary node.

Please advice how can make atlleast one secondary node replicating with primary.

Hi,
I would like to recommend you to upgrade your MongoDB database, version 2.6.12 is very old and out of support.
Regarding you replica set question - the replica set member becomes “stale” when its replication process falls so far behind that the primary overwrites oplog entries the member has not yet replicated. When this occurs, you must completely resynchronize the member by removing its data and performing an initial sync.

please read Resync a Member of a Replica Set

  1. stop mongod instance on “stale” secondary
  2. delete all data and sub-directories from the member’s dbPath directory.
  3. start mongod process on “stale” secondary
    At this point, the mongod will start the initial sync, and when the initial sync will finish mongod replica set member becomes a secondary
2 Likes

Thank you very much for your quick response.
This worked in a lower environment where dbpath is less size.
But incase of production, we have 5.2TB of data to replicate from primary if we perform initial sync.

Do we any other options. ?

Thanks Again,

you can also Sync by Copying Data Files from Another Member, I think a snapshot can be a good solution.

2 Likes

We started resync process on one the secondary node. It was 50 percent complete and stuck for long time and not progressing further. It was in startup2 mode.
So we restarted the mongod service on that node. Then mongod service came and started syncup from the beginning again. I could not see the earlier synced up files.
Can you please suggest on this issue.

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.