I am using mongodump to back up the entire database, which is around 200 GB in size. My command looks like this: mongodump --uri=theuri -d=dbName -o=outputPath --gzip --numParallelCollections=10.
After the dump is complete, I have a script that decompresses the gzipped files back to JSON format, removes the _id index from all *.metadata.json files, and then encodes them back to gz format. This process is necessary to address the E11000 error that occurs when I try to reimport using mongorestore. If I don’t remove the _id index, the restore process throws the E11000 error and only a few hundred thousand out of the 70 million documents are imported successfully.
The restore process itself works fine for all collections, and all documents are reimported correctly without any failures. However, I have encountered an issue where the views I created in the original database are converted into collections after reimporting from the backup. I’m currently trying to understand why this is happening. I suspect it’s because I am changing the metadata.json? I only edit the indexes field of the metadata; Source: https://github.com/legendhimself/Mongo_RustBackup/blob/main/src/utils/mod.rs#L80;
Here’s the command for restore
mongorestore --uri="" --gzip --numParallelCollections 10 --numInsertionWorkersPerCollection 10 -d Name ./
Output: 2023-08-01T03:27:19.666+0530 73379560 document(s) restored successfully. 0 document(s) failed to restore.