I was having an “Incorrect number of documents imported” error after running the validator before stumbling across this thread and after multiple re-imports, here are my findings.
From my tests, it turns out that even though
sh.status() shows the number of chunks between the shards, it’s actually still doing some work in the background, i.e. sharding is not truly complete. I believe that what the
sh.status shows is the way it’s going to split the data but it doesn’t truly confirm that all the work to split the chunks is complete. I’m sure that with a much more powerful server, you won’t notice this background process.
- You don’t need to drop
applicationData db because it’s not interfering. It just so happens that during the process of dropping this db, the allocation of chunks actually completes. I can’t imagine that there will be a clash of collection names in an enterprise grade database like MongoDB.
- You don’t need to run a loop against the validator. Again, it just so happens that during the process of looping, the allocation of chunks actually completes.
- You don’t need to drop the m103 db using
--drop command in
mongoimport takes care of that.
Here are my workings/solution approach:
- Connect to both replica sets and run a products count on both:
mongo --host “m103-repl/m103:27001” -u “m103-admin” -p “m103-pass” --authenticationDatabase “admin”
mongo --host “m103-repl-2/m103:27004” -u “m103-admin” -p “m103-pass” --authenticationDatabase “admin”
- If you find that one of the replica set has a count of 516784, the sharding process is not yet complete. You may find that even though one replica set has a count of 516784, the other replica set may return a count of 200K+… the sharding process is still running.
- Keep checking the count on both replica sets until the sum of the count of products in both replica sets equals the total number of products
- Finally, run the validator. You may encounter a timeout issue when you run the validator, just re-run it.
These are my thoughts after spending an hour trying to get to the bottom of this.
Curriculum Engineers, please feel free to chime in.