Adding new member with copy of data folder doesn't sync

[Nivi_Mor]

Hi,

I want to add a new member to an existing RS. I understand that since there’s a lot of data in the cluster, initial sync will take a long time.

So followed the instructions here - Add Members to a Replica Set — MongoDB Manual
which suggests copying the data folder from an existing member. However, when I start the node and add it to the cluster, it performs initial sync and is stuck in STARTUP2 status. Looking at the rs.printSlaveReplicationInfo() I see:

	syncedTo: Thu Jan 01 1970 00:00:00 GMT+0000 (UTC)
	1647860466 secs (457739.02 hrs) behind the primary

The docs say that I should make sure that I “can copy the data directory to the new member and begin replication within the window allowed by the oplog. Otherwise, the new instance will have to perform an initial sync, which completely resynchronizes the data”.

The oplog is configured at 409600MB. The data copied is recent.

What else should I look for or try?

[Nivi_Mor]

I found the solution, if anyone comes across the same issue:

1. The member from which you copy the data or take a snapshot must be off as well. Shutdown the DB safely and copy the data or take a snapshot.
2. The new member must be turned off, that is to say mongod should not run on the new member host.
3. Copy the data or apply the snapshot such that the data is stored in the folder of the mongo config dbPath.
4. Before starting mongod on the new member, make sure that the data folder is owned by mongodb user.
5. Before starting mongod on the new member, make sure that it has the same replication config as the member from which you copied the data.
6. Turn on the member and let it apply the opLog. You can run sudo tail -f /var/log/mongodb/mongodb.log (or whatever your log path is for mongo) to track the progress.
7. When the new member finishes applying the oplog, you should see something like Locally stored replica set configuration does not have a valid entry for the current node; waiting for reconfig or remote heartbeat; Got "NodeNotFound: No host described in new configuration 1 for replica set shard0 maps to this node" while validating - that’s ok. The new member says “I’m part of this replicaset, but my host is not listed in the replicaset collection”.
8. Add the new member via the primary member - rs.add(“HOST:27017”).

Good luck!