Mongorestore gives not master error

We backed up mongodb data using mongodump. When we tried to restore the data using mongorestore, sometimes we get this error - : insertion error: not master. Our logic has two retries if there are errors in the first attempt. At the end, we found that this error shows for a particular collection’s restore in all three attempts. I read online posts, people pointed out this error is usually due to the write operation on the secondary. But if that is the cause, how come the restore operations on other collections were successful? I am not convinced this is the cause in our restore process. Can anyone shed lights on why mongorestore gives this error? Thanks.

log messages -

2023-10-19T22:32:23.866+0000 finished restoring OAuthDBSchema.OauthClient (2 documents)
2023-10-19T22:32:23.866+0000 reading metadata for platform-db.Team from /mongo-br/last/platform-db/Team.metadata.json
2023-10-19T22:32:23.866+0000 restoring platform-db.Team from /mongo-br/last/platform-db/Team.bson
2023-10-19T22:32:23.866+0000 finished restoring OAuthDBSchema.OauthToken (299 documents)
2023-10-19T22:32:23.866+0000 reading metadata for platform-db.ScimAttributeMapping from /mongo-br/last/platform-db/ScimAttributeMapping.metadata.json
2023-10-19T22:32:23.866+0000 restoring platform-db.ScimAttributeMapping from /mongo-br/last/platform-db/ScimAttributeMapping.bson
2023-10-19T22:32:23.882+0000 Failed: platform-db.Users: error restoring from /mongo-br/last/platform-db/Users.bson: insertion error: not master

Hi @Yanni_Zhang and welcome to MongoDB community forums!!

As per my understanding,

the write operation on Secondary node is not allowed and would give you the error as:

connected to: mongodb://localhost:2001/
2023-10-26T16:34:39.576+0530	Failed: (NotWritablePrimary) not primary

In order to understand the issue further, could you help me with some information as mentioned below:

  1. The deployment type you are on(e.g. no of secondary, arbiters etc)
  2. The output for rs.status() or rs.conf()
  3. The MongoDB version you are on.
  4. A code snippet that would help me reproduce the error.
  5. Can you help me with the mongodump and mongorestore command that you are using.
  6. Finally, could you confirm if there is any operation performed on the configuration of the replica set while performing the restore operation.

Regards
Aasawari

Hi, Aasawari,

The error was seen when we ran the mongorestore command.

  1. The deployment type you are on(e.g. no of secondary, arbiters etc)
    There is one primary pod and 2 secondary pods, no aribter.

  2. The output for rs.status() or rs.conf()
    We did not capture it.

  3. The MongoDB version you are on.
    4.0.25

  4. A code snippet that would help me reproduce the error.
    We observed it in restore process. One collection’s restore out of all had this problem. It’s not always reproducible. We just use mongodump https://www.mongodb.com/docs/database-tools/mongodump/ for backup, and use mongoretore mongorestore — MongoDB Manual for restoring it to another mongo instance.

  5. Can you help me with the mongodump and mongorestore command that you are using.
    See above

  6. Finally, could you confirm if there is any operation performed on the configuration of the replica set while performing the restore operation.

No configuration on the replica set.

Hi @Yanni_Zhang and thank you for writing back.

The MongoDB version mentioned above quiet old and was released around June 2021.

MongoDB 4.0 is no longer supported, and upgrading to a newer version can provide improved stability, bug fixes, and additional features. You can refer to the EOL Support Policies for more information on MongoDB versions and their support status.

However, based on the error message, it appears that you are connected to a secondary node that cannot accept write operations. I would suggest verifying the current primary node and then trying the mongorestore operation again.

Also please let us know if you are still seeing the issue consistently after the upgrade.

Best Regards
Aasawari