4 node replica set + 1 arbiter. What happens when 2 data nodes go down?

If you have have replicaset with 4 data nodes and 1 arbiter, what is the expected behavior if 2 data nodes are down? will the remaining 2 data-nodes + arbiter elect one of the data nodes as a primary so the cluster can still operate?

A primary can be elected, since majority (3 nodes) can still be met. (arbiter can vote). But majority writes can not be performed as at most 2 can ack on a write (arbiter can’t ack any write).

4 Likes

Important but easy to miss. I know, I did.

1 Like

Thanks! The sentence you wrote should be put as an important note on the MongoDB docs!

Hi @AmitG,

There are several warnings related to arbiters in the MongoDB documentation.

Under Add an Arbiter to a Replica Set:

If you are using a three-member primary-secondary-arbiter (PSA) architecture, consider the following:

  • The write concern "majority" can cause performance issues if a secondary is unavailable or lagging. For advice on how to mitigate these issues, see Mitigate Performance Issues with PSA Replica Set.
  • If you are using a global default "majority" and the write concern is less than the size of the majority, your queries may return stale (not fully replicated) data.

There is also a warning under Calculating Majority for Write Concern:

There is more detail in the documentation links above, but in general it is better to avoid the use of arbiters where possible.

Regards,
Stennie

1 Like

Thanks! I had actually read that section a couple times but it was not clear to me what the ramifications would be for a 4-1 replicaset. The post from Kobe succinctly described the downside in a concrete way that I could understand. One of our other lead engineers missed the downstream effect of majority write concern even though a primary can be elected.

Thanks for the clarification on avoiding arbiters as much as possible!

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.