Memory issue : migrating database from standalone to replica set

Ops and Admin
1

currently we are using standalone mongodbInstance with 2core 8gb ram.
we have planned to move to replicaSet(3Node replica) based setup to handle more number of readWrite requests.
i exported data using mongodump. Among these exported collections, some collections has size >400mb, some have 800mb.

when i import using mongorestore command, i get runtime outofmemory error.
iam trying this out on 4core8gb ram Instance.

To fix this problem , i thought that rate of which the collections are imported + replication happening at the same time can cause this issue. So i wrote a script that can restore all the collection one by one but with some delay.

The intention behind having this delay is, this delay will give sufficient time to load a collection and once loaded, the script will pause for sometime, this will allow replication to secondaryNodes.This could freeup ram while loading the nextCollection. And i have added larger delay’s for larger collections.

But even at this approach, the ram gets too low. And it takes lot of time to load all the collections due to the delay that i have set.

is there a standard way to approach this problem in mongodbReplicaSet? Please help.

2

You should have a look at the mongod logs too, you may be on the edge of catastrophe.

Follow this to convert the standalone to a replica set.

Once you have a replica set configured remember to update the clients connection strings/parameters.

3

i checked the links but iam not able to find any details to restoring collections with high volume.

the issue iam facing right now goes like this

  • i have a standalone MongoDBInstance running on machine1.
  • i have configured the MongoDBReplicaSet nodes on machine2,machine3,machine4.
  • This replica set is running fine by itself.
  • no issues so far.
  • Now, i dumped data present in MongoDBInstance(Machine1) into MongoDBNode present in Machine2(primaryNode).multiple Collections with size >400mb,>800mb are present in this dump.
  • no issues so far.
  • i run mongorestore by pointing to the primaryNode - i get runtime-outofmemory error.
  • i created a script (its present in my question) to handle this issue and to restore data succesfully.
  • here is my query. is there a standard approach to resolve this issue occuring during mongorestore?