Manually change hidden node to primary when all other nodes fail

We have a setup MongoDB cluster that consists of a total of 5 nodes of which 3 are on-premise and 2 nodes on cloud server. We have configured 2 nodes on cloud servers to be hidden and priority-0 to avoid delays in writes due to latency.

In case of an outage at our DC, the only nodes that will remain in our cluster will be the hidden nodes that are on the cloud. In such a scenario, can we somehow make the hidden nodes primary and secondary? And later add more nodes to it?

1 Like

Hi @Map_Sec

The procedure is outlined in reconfigure-a-replica-set-with-unavailable-members. The remaining members will need priority and hidden correctly set too. You will be wanting to add another member immediately to avoid running with a two node replicaSet.

As these nodes are hidden any connection uris would have to be updated as the client drivers would not receive the topology update when the replicaSet is reconfigured.

You definitely want to test this procedure in a non-prod environment before you need to execute this.

Unless you are changing votes as well, hidden members will still take part in writeConern acknowledgment so your experience may not match you expectations if one or two nodes in the DC become unavailable.

2 Likes

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.