I am having a standalone server with 50TB data in it with multiple database. I am not allowed to down the server for backup process. How I can take a partial & incremental backup of such server using any script. I already find out there were option to take backup of replica set but didnt found anything for standalone server.
My goal is to create a script which take dump of chunk of data one by one till the current time & then start the incremental backup with specific interval.
Appreciate the help. I am flexible with all type of scripts like python, shell script, etc.
I think you may need to reconfigure your standalone instance as a single-node replicaset. It will require a restart, but that should be all. I’ll admit that for a database of this scale, I don’t know the impact of this change. (But it shouldn’t be anything like as bad as taking down the database to copy all of the data!)
Once you have a replicaset, and thus an oplog, the initial backup can be performed with mongodump and the --oplog flag - that way, as well as taking a full backup of the data, the oplog will be copied as well, so at the point the dump completes, you’ll have a full copy of the dataabase up to the exact time the command completed. The docs for that are here: https://www.mongodb.com/docs/manual/tutorial/backup-and-restore-tools/#create-backups-using-oplogs
Do be aware that running mongodump on all your data can have some pretty significant performance impacts.
I’m not entirely certain how you’d follow up that initial backup with incremental backups, TBH, without upgrading to MongoDB Enterprise, which specifically includes tools for managing backups.
While we’re talking about this, if you have 50TB of data in a single instance that’s so important you can’t take it down for a backup - that’s a huge red flag. I’m guessing you already know this By transitioning to the recommended 3 instance minimum replicaset you would reduce the risk of downtime significantly. If your current instance goes down you’re obviously going to be down for some time while you reload data into it.
By transitioning to the recommended 3 instance minimum replicaset you would reduce the risk of downtime significantly. If your current instance goes down you’re obviously going to be down for some time while you reload data into it.