Migrating MongoDB data from on premise to Azure VM

Ops and Admin
1

Hi,

I am trying to migrate mongodb data from On Premise server to Azure VM using replica set. The version of MongoDB is 2.6.12 and the data is about 6.5 TiB.

For one of the server which had less amount of data, replication was successful but for this server, the replica set on secondary node is not acquiring secondary state and I am getting error as “Too many files open”.

I have also set the limit for open files to 700000000, core file size to unlimited, file size to unlimited, max memory size to unlimited, max locked memory to unlimited, virtual memory to unlimited and file locks to unlimited but all in vain…

After making all the changes to the replica set in secondary node, I always get an error of too many open files on reaching to a point at 1937 GB.

Please suggest me of some ways to make this data get migrated from on premise to Azure VM.

2

This is ancient. Current versions are 4.4, 5.0, 6.0.
4.4 is EoL February 2024
5.0 is EoL October 2024

So you should target an upgrade to 6.0

It does not appear they are getting set in the correct places. Check they are being set in the init/systemd script that is invoking mongod.

Check the running limits of mongod: cat /proc/$(pgrep -x mongod)/limits to confirm if they are set or not.

3

We can’t upgrade now. Upgradation is planned after the data migration and go live in cloud.

4

This will be an epic upgrade to a current version!

Lots of upgrade notes and many major changes to take care of along the way. And don’t forget the application drivers.
2.6 → 3.0 → 3.2 → 3.4 → 3.6 → 4.0 → 4.2 → 4.4

1 Like
5

Can you please let me know of some way to replicate the data from my current on premise version to cloud…

I know the upgrade will be epic😜

6

You’re on the right course adding a replica. You need to see why the limits are not being set, my guess is they are not being set correctly in the systemd/upstart/init.

But the output of the command in my previous post will confirm that.