Mongorestore fails to copy/clone an existing database to a new one

Working with Data Developer Tools
,
1

I have MongoDB version 5.0.7 installed in the default location in my PC running on Windows 10 Pro. I have a database named mydb which contains a collection called Products. I want to copy the mydb database into a new database called newdb. For that, I’m trying to use mongodump and mongorestore commands as mentioned here. While doing so, although mongodump is executing and creating the dump file properly, the mongorestore command fails everytime generating a duplicate key error.

The command I use to dump the mydb database to an archive:
mongodump --archive="mongodump-mydb" --db=mydb

The command I use to restore the database from the dump to a new database:
mongorestore --archive="mongodump-mydb" --nsFrom='mydb.*' --nsTo='newdb.*'

The error message that I get:

2022-04-17T19:09:04.136+0530    preparing collections to restore from
2022-04-17T19:09:04.146+0530    reading metadata for mydb.products from archive 'mongodump-mydb'
2022-04-17T19:09:04.150+0530    restoring to existing collection mydb.products without dropping
2022-04-17T19:09:04.157+0530    restoring mydb.products from archive 'mongodump-mydb'
2022-04-17T19:09:04.209+0530    continuing through error: E11000 duplicate key error collection: mydb.products index: _id_ dup key: { _id: ObjectId('625974fe025923931b6e5e7f') }
2022-04-17T19:09:04.211+0530    continuing through error: E11000 duplicate key error collection: mydb.products index: _id_ dup key: { _id: ObjectId('62597508025923931b6e5e80') }
2022-04-17T19:09:04.211+0530    continuing through error: E11000 duplicate key error collection: mydb.products index: _id_ dup key: { _id: ObjectId('62597508025923931b6e5e81') }
2022-04-17T19:09:04.211+0530    finished restoring mydb.products (0 documents, 3 failures)
2022-04-17T19:09:04.211+0530    no indexes to restore for collection mydb.products
2022-04-17T19:09:04.211+0530    0 document(s) restored successfully. 3 document(s) failed to restore.

Surprisingly, it seems to restore the collection products in mydb database instead of newdb. As mydb already contains the products collection having rows of documents with unique ids, it’s creating a clash generating the duplicate key error.

As an alternative, while executing mongorestore I put the name of the new database in both --nsFrom and --nsTo arguments to check if it makes some difference:

mongorestore --archive="mongodump-mydb" --nsFrom='newdb.*' --nsTo='newdb.*'

Yet, it generates the same error.

Somehow, mongo seems to be ignoring the new database argument altogether. Why is this happening and how do I solve this problem? Because, copying existing data from a database into a new one is a common requirement of my project, and if I can’t restore my data from dump, all it takes is a crash to loose my records forever. So please help me in resolving this issue.

Regards

2

I found the reason. The problem lies in using single quotes while passing the values of --nsFrom and --nsTo arguments. While doing so, in some environments, mongo keeps on restoring the dump to the existing database instead of the new one, thus creating a clash between the existing data and the data being restored generating the “duplicate key error”. When you encounter such a scenario, either use double quotes to pass the values of --nsFrom and --nsTo arguments or don’t use any quote at all.

This definitely seems to be a bug in MongoDB Server system or its syntax. MongoDB development team must take note of this and rectify it in the next update.

1 Like
3

This is definitively not a but in MongoDB server system or syntax. The mongo commands do not receive the quotes. The quotes are interpreted by the calling shell.

See

4

If the mongo commands don’t receive the quotes, then they shouldn’t be there in the documentation. When it’s written in the documentation, any user will try to implement it for testing and thus, if it generates a bunch of bewildering error messages that are somehow unrelated to the objective of the user, it becomes quite confusing.

What’s even more confusing is that the same command runs properly with quotes on the arguments on a Mac machine; but it doesn’t when run on a Windows PC.

If there’s some shell interpretation issue involved, the documentation on mongodump and mongorestore should clearly specify that at the exact place. I don’t know if I missed it somewhere, but I couldn’t see it. Otherwise, it ends up being puzzling for readers, especially the beginners.

5

It is the chicken and egg problem.

If they do not put quotes in the examples, most examples will fail in all CLI rather than in a few.

They also have to assume some prior knowledge and cannot explained the quotes, variable expansions, path settings and other subtleties of each CLI. There are too many. Even if they could, I would hate have to scroll over those shell, CLI details all the time when I use the documentation.

I feel it is the responsibility of the CLI user to know the details and particularities of the environment he uses. Yes there is a learning curve. Yes there are frustrations. I have a lot each time I am forced to use PowerShell because I am a Unix guy, so I feel you pain. I just think there is no easy solution.

Try to enjoy the tools for the service they provide despite the frustration.

6

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.