Issues at adding an arbiter in a PSA configuration

Hi, guys. I’m using MongoDB 5.0.5 hosted on RHEL7 servers. I tried to deploy a PSA configuration, but I was only succeeded in the PS part: adding the arbiter has been very difficult. When I thought I got it, after the addArb command, rs.status() output shows me the following message, and I will show you the most important part of it:
{

stateStr: ‘(not reachable/healthy)’,

lastHeartbeatMessage: ‘Error connecting to hl_arbiters.dops.etecsa.cu:27056 (172.16.1.179:27056) :: caused by :: Permission denied’,

}

Also, I got this meesage from the arbiter Mongod instance’s log, even before I run the addArb command:

“ctx”:“monitoring-keys-for-HMAC”,“msg”:“Failed to refresh key cache”,“attr”:{“error”:“NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.”,“nextWakeupMillis”:7600}}

I tried almost everything, from configure the firewall in RHEL7 servers to customize SELinux, because I used an unusual port for the arbiter (I want to deploy two arbiters, really, that’s the reason why I didn’t use 27017). Nothing changed, I still cannot deploy my PSA architecture. What am I doing wrong? Thanks in advance.

Why do you want to add two arbiters.You should have only one arbiter
From mongo docs

WARNING
** In general, avoid deploying more than one arbiter per replica set.**

May be nodes are unable to communicate on port 27056
Did you try with 27017 with single arbiter?
Also how was arbiter started like bindIp params etc
Do you see waiting for connections in the log?

1 Like

Thanks for your response, Ramachandra. Well, there will be two arbiters because I pretend to work with two different sets of Replica Sets, I mean, one arbiters will perform voting in one 2-nodes ReplicaSet, this one will carry heavy loads, and the other arbiter will perform voting in another 2-nodes ReplicaSet, this one with light loads. Nodes can communicate on port 27056, I mean, I can connect from my laptop, and from any of the hosts involved in the ReplicaSet to the mongod used as an arbiter, which is in another host. I didn’t see then, and I don’t see now wating for connections messages in the log, just the messages I’ve written in the previous, and first, post.
Reading while trying to find out what’s happening, I found some blogs with different opinions about using an arbiter. SOme of’em even discourage its use. I really test to shutdown the PRIMARY, and after a few seconds, SECONDARY turns PRIMARY and as long as I’m only using a PS, maybe I don’t really need a PSA architecture. Nevertheless, from the API I insert documents, I always need a connection to Mongo, in this case, as all my writes will be directed to the PRIMARY one, then I need to be connected to the PRIMARY, and it means, I will write down its URL in the connection string, right? What about this scenario: PRIMARY goes down, and as I can checked yesterday, SECONDARY turns PRIMARY, but my daemon, the one which inserts documents, has the IP of the former PRIMARY host. WHat should I do? wWTry to insert, get the exception, change the connection?. Thanks again.

Dear Ramachandra, about the last part of my reply, the one which I asked about connecting to a Replica Set, I found the answer in pymongo readthedosc website. The same topic I found in Java API to connecto to MongoDB, also in Rust. So, thanks in advance. ABout the first part, everything keeps in the same way: same messages, the same “Permission denied”

It could be firewall or bindIP issue.I asked about it in my first reply

https://jira.mongodb.org/browse/SERVER-23363