Unable to add nodes into Replicaset

Hi ,
Am unable to add secondary node into replica set .getting below error.Can anyone help.

{
“operationTime” : Timestamp(1654863211, 1),
“ok” : 0,
“errmsg” : “Either all host names in a replica set configuration must be localhost references, or none must be; found 1 out of 2”,
“code” : 103,
“codeName” : “NewReplicaSetConfigurationIncompatible”,
“$clusterTime” : {
“clusterTime” : Timestamp(1654863211, 1),
“signature” : {
“hash” : BinData(0,“AAAAAAAAAAAAAAAAAAAAAAAAAAA=”),
“keyId” : NumberLong(0)
}
}

Hi @Rojalin_Das1 and welcome in the MongoDB Community :muscle: !

Error message seem pretty clear. Can you share your cluster config and how you are trying to add the new node?

Cheers,
Maxime.

Below is the config file

mongod.conf

for documentation of all options, see:

http://docs.mongodb.org/manual/reference/configuration-options/

where to write logging data.

systemLog:
destination: file
logAppend: true
path: /database/log/mongodb/mongod.log

Where and how to store data.

storage:
dbPath: /database/data/mongodb
journal:
enabled: true

engine:

mmapv1:

wiredTiger:

how the process runs

processManagement:
fork: true # fork and run in background
pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo

network interfaces

net:
port: 27019
bindIp: 127.0.0.1,hostname # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.

#security:

#operationProfiling:

#replication:
replication:
replSetName: “SNT-NP-DEV-rs01”

#sharding:

Enterprise-Only Options

#auditLog:

#snmp:

One thing to share here, while doing rs.status(), host name showing as localhost not the actual hostname.
“members” : [
{
“_id” : 0,
“name” : “localhost:27019”,
“health” : 1,
“state” : 1,
“stateStr” : “PRIMARY”,
“uptime” : 5876,
“optime” : {
“ts” : Timestamp(1654867891, 1),
“t” : NumberLong(1)
},

Adding node: rs.add(“hostname:27019”)

The problem was when you did the first rs.init(). I guess you didn’t provide a config there and it took localhost by default.

Send a new config with the correct hostname or IP address. Then add the new node.

I didn’t get you to send a new config with the correct hostnam. Could you please elaborate it

You have to use the rs.reconfig() command like below. In this example I change “localhost” to “hafx” which is my actual hostname that other computers on my network can find.

test [direct: primary] test> conf = rs.conf()
{
  _id: 'test',
  version: 2,
  term: 1,
  members: [
    {
      _id: 0,
      host: 'localhost:27017',
      arbiterOnly: false,
      buildIndexes: true,
      hidden: false,
      priority: 1,
      tags: {},
      secondaryDelaySecs: Long("0"),
      votes: 1
    }
  ],
  protocolVersion: Long("1"),
  writeConcernMajorityJournalDefault: true,
  settings: {
    chainingAllowed: true,
    heartbeatIntervalMillis: 2000,
    heartbeatTimeoutSecs: 10,
    electionTimeoutMillis: 10000,
    catchUpTimeoutMillis: -1,
    catchUpTakeoverDelayMillis: 30000,
    getLastErrorModes: {},
    getLastErrorDefaults: { w: 1, wtimeout: 0 },
    replicaSetId: ObjectId("62a35084f93f180c51796d9d")
  }
}
test [direct: primary] test> conf.members[0].host = 'hafx:27017'
hafx:27017
test [direct: primary] test> rs.reconfig(conf)
{
  ok: 1,
  '$clusterTime': {
    clusterTime: Timestamp({ t: 1654870331, i: 1 }),
    signature: {
      hash: Binary(Buffer.from("0000000000000000000000000000000000000000", "hex"), 0),
      keyId: Long("0")
    }
  },
  operationTime: Timestamp({ t: 1654870331, i: 1 })
}
test [direct: primary] test> rs.conf()
{
  _id: 'test',
  version: 3,
  term: 1,
  members: [
    {
      _id: 0,
      host: 'hafx:27017',
      arbiterOnly: false,
      buildIndexes: true,
      hidden: false,
      priority: 1,
      tags: {},
      secondaryDelaySecs: Long("0"),
      votes: 1
    }
  ],
  protocolVersion: Long("1"),
  writeConcernMajorityJournalDefault: true,
  settings: {
    chainingAllowed: true,
    heartbeatIntervalMillis: 2000,
    heartbeatTimeoutSecs: 10,
    electionTimeoutMillis: 10000,
    catchUpTimeoutMillis: -1,
    catchUpTakeoverDelayMillis: 30000,
    getLastErrorModes: {},
    getLastErrorDefaults: { w: 1, wtimeout: 0 },
    replicaSetId: ObjectId("62a35084f93f180c51796d9d")
  }
}

Final rs.status()

test [direct: primary] test> rs.status()
{
  set: 'test',
  date: ISODate("2022-06-10T14:12:21.947Z"),
  myState: 1,
  term: Long("1"),
  syncSourceHost: '',
  syncSourceId: -1,
  heartbeatIntervalMillis: Long("2000"),
  majorityVoteCount: 1,
  writeMajorityCount: 1,
  votingMembersCount: 1,
  writableVotingMembersCount: 1,
  optimes: {
    lastCommittedOpTime: { ts: Timestamp({ t: 1654870331, i: 1 }), t: Long("1") },
    lastCommittedWallTime: ISODate("2022-06-10T14:12:11.371Z"),
    readConcernMajorityOpTime: { ts: Timestamp({ t: 1654870331, i: 1 }), t: Long("1") },
    appliedOpTime: { ts: Timestamp({ t: 1654870331, i: 1 }), t: Long("1") },
    durableOpTime: { ts: Timestamp({ t: 1654870331, i: 1 }), t: Long("1") },
    lastAppliedWallTime: ISODate("2022-06-10T14:12:11.371Z"),
    lastDurableWallTime: ISODate("2022-06-10T14:12:11.371Z")
  },
  lastStableRecoveryTimestamp: Timestamp({ t: 1654870319, i: 1 }),
  electionCandidateMetrics: {
    lastElectionReason: 'electionTimeout',
    lastElectionDate: ISODate("2022-06-10T14:09:08.971Z"),
    electionTerm: Long("1"),
    lastCommittedOpTimeAtElection: { ts: Timestamp({ t: 1654870148, i: 1 }), t: Long("-1") },
    lastSeenOpTimeAtElection: { ts: Timestamp({ t: 1654870148, i: 1 }), t: Long("-1") },
    numVotesNeeded: 1,
    priorityAtElection: 1,
    electionTimeoutMillis: Long("10000"),
    newTermStartDate: ISODate("2022-06-10T14:09:09.484Z"),
    wMajorityWriteAvailabilityDate: ISODate("2022-06-10T14:09:09.565Z")
  },
  members: [
    {
      _id: 0,
      name: 'hafx:27017',
      health: 1,
      state: 1,
      stateStr: 'PRIMARY',
      uptime: 197,
      optime: { ts: Timestamp({ t: 1654870331, i: 1 }), t: Long("1") },
      optimeDate: ISODate("2022-06-10T14:12:11.000Z"),
      lastAppliedWallTime: ISODate("2022-06-10T14:12:11.371Z"),
      lastDurableWallTime: ISODate("2022-06-10T14:12:11.371Z"),
      syncSourceHost: '',
      syncSourceId: -1,
      infoMessage: '',
      electionTime: Timestamp({ t: 1654870148, i: 2 }),
      electionDate: ISODate("2022-06-10T14:09:08.000Z"),
      configVersion: 3,
      configTerm: 1,
      self: true,
      lastHeartbeatMessage: ''
    }
  ],
  ok: 1,
  '$clusterTime': {
    clusterTime: Timestamp({ t: 1654870331, i: 1 }),
    signature: {
      hash: Binary(Buffer.from("0000000000000000000000000000000000000000", "hex"), 0),
      keyId: Long("0")
    }
  },
  operationTime: Timestamp({ t: 1654870331, i: 1 })
}

Thanks. It woked.
Shall i follow same process for other nodes as well ?

So that i can add member into replica set.

Thanks a lot . My problem is solved. I have created Replica set successfully.

When you send the rs.add(XXX) command, the hostname and port that you send will be used to add the node in the config. Make sure to use the right hostname / IP address when you add them and you won’t have to reconfigure.

Next time, directly pass the right config when you send the rs.initiate(xx) command:

Something like this for example:

{
      _id: "replicaSetName",
      members: [
         { _id: 0, host: "hostnameX:27017" },
         { _id: 1, host: "hostnameY:27017" },
         { _id: 2, host: "hostnameZ:27017" }
      ]
}

Cheers,
Maxime.

Hi ,
Again having issue today to add nodes into replica set .
I stopped mongod services and while starting all nodes went secondary .To make one node primary, i have removed all nodes except one node by command(conf.members.splice(1,2) ).
Then that node automativally come to primary node and when i start to add other nodes, its not adding now.Getting below error. Please help on this issue.

Error:
SNT-DEV-rs01:PRIMARY> rs.add(“hostname:27019”)
{
“operationTime” : Timestamp(1655109480, 1),
“ok” : 0,
“errmsg” : “Quorum check failed because not enough voting nodes responded; required 2 but only the following 1 voting nodes responded: hostname:27019; the following nodes did not respond affirmatively: hostname:27019 failed with Error connecting to hostname:27019 (10.36.7.42:27019) :: caused by :: Connection refused”,
“code” : 74,
“codeName” : “NodeNotFound”,
“$clusterTime” : {
“clusterTime” : Timestamp(1655109480, 1),
“signature” : {
“hash” : BinData(0,“AAAAAAAAAAAAAAAAAAAAAAAAAAA=”),
“keyId” : NumberLong(0)
}
}
}

Did you replace hostname with actual hostname in your rs.add command?
Please show output of
hostname
Also did you try to add nodes using IP?

Please please please, do not use hostname as your host name. We usually use hostname as place holder to indicate that a real host name should be specified. Using hostname as an hostname is very confusing and error prone.

What is funny in your error message is that it says that only hostname:27019 responded and at the same time it says that hostname:27019 did not respond.

This makes me think that hostname, did I say it is a bad choice, does not resolve to the same IP in all of your hosts.

Is there any reasons why you are not using the default port number?

There is no reason not to use default port but i want to use different port. Regarding error msg, i have just replace server name as “hostname” and in all place server name is same.

Not sure why the error coming. Could you please guide me what necessary thing i can check and correct.

Hi,
Due to errors,i have uninstalled mongod package and installed it again in servers but while doing initiate of replicaset it thowing below error.Am unable to initiate replicaset by passing all host details which command advised by you earlier. Kindly help me to correct it.

@(shellhelp2):1:1

rs.initiate({
… _id : “rs0”,
… members: [
… { _id : 0, host: “abc0a:27019” },
… { _id : 1, host: “abc1a:27019” },
… { _id : 2, host: “abc2a:27019” }
… ]
… })
{
“operationTime” : Timestamp(0, 0),
“ok” : 0,
“errmsg” : "replSetInitiate quorum check failed because not all proposed set members responded affirmatively: abc1a:27019 failed with Error connecting to abc1a:27019 (XX.XX.X.XX:27019) :: caused by :: Connection refused, abc2a:27019 failed with Error connecting to abc2a:27019 (XX.XX.X.XX:27019) :: caused by :: Connection refused,
“code” : 74,
“codeName” : “NodeNotFound”,
“$clusterTime” : {
“clusterTime” : Timestamp(0, 0),
“signature” : {
“hash” : BinData(0,“AAAAAAAAAAAAAAAAAAAAAAAAAAA=”),
“keyId” : NumberLong(0)
}
}
}

HI,
Due to different errors,i did mongo package uninstallation and installed it again.And while initiate replicaset by passing all host name am getting below error.

rs.initiate({
… _id : “rs0”,
… members: [
… { _id : 0, host: “abc0a:27019” },
… { _id : 1, host: “abc1a:27019” },
… { _id : 2, host: “abc2a:27019” }
… ]
… })
{
“operationTime” : Timestamp(0, 0),
“ok” : 0,
“errmsg” : "replSetInitiate quorum check failed because not all proposed set members responded affirmatively: abc1a:27019 failed with Error connecting to abc1a:27019 (XX.XX.X.XX:27019) :: caused by :: Connection refused, abc2a:27019 failed with Error connecting to abc2a:27019 (XX.XX.X.XX:27019) :: caused by :: Connection refused,
“code” : 74,
“codeName” : “NodeNotFound”,
“$clusterTime” : {
“clusterTime” : Timestamp(0, 0),
“signature” : {
“hash” : BinData(0,“AAAAAAAAAAAAAAAAAAAAAAAAAAA=”),
“keyId” : NumberLong(0)
}
}
}

HI All,

My issue has been resolved after starting mongod with config file. Replication has done sucessfully

Thank You

1 Like