Replication status shows lastCommittedOptime as 0

Hi,

I have a replica set on which replication was going fine. Suddenly I am seeing below value in replication status output and looks like there is an issue with replication. What might be causing it and any resolution for this please?

optimes" : { “lastCommittedOpTime” : { “ts” : { “$timestamp” : { “t” : 0, “i” : 0 } }, “t” : -1 },

Thanks,
Akshaya Srinivasan

Welcome @Akshaya_Srinivasan!

To help investigate this problem can you please share:

  • the specific version(s) of MongoDB server used in your replica set
  • the roles of the members showing lastCommittedOpTime set to 0
  • your rs.status() output from the current primary with the hostnames redacted

Thanks,
Stennie

1 Like

Hi Stennie,

  • the specific version(s) of MongoDB server used in your replica set --> MongoDB 3.4.5
  • the roles of the members showing lastCommittedOpTime set to 0 --> I do not have authentication enabled on this server. Server is started and access as Windows administrator user.
  • your rs.status() output from the current primary with the hostnames redacted
    myShard_0:PRIMARY> rs.status()
    {
            "set" : "myShard_0",
            "date" : ISODate("2020-03-16T06:53:24.078Z"),
            "myState" : 1,
            "term" : NumberLong(57),
            "heartbeatIntervalMillis" : NumberLong(2000),
            "optimes" : {
                    "lastCommittedOpTime" : {
                            "ts" : Timestamp(0, 0),
                            "t" : NumberLong(-1)
                    },
                    "appliedOpTime" : {
                            "ts" : Timestamp(1584341601, 1),
                            "t" : NumberLong(57)
                    },
                    "durableOpTime" : {
                            "ts" : Timestamp(1584341601, 1),
                            "t" : NumberLong(57)
                    }
            },
            "members" : [
                    {
                            "_id" : 0,
                            "name" : "xxxx:27017",
                            "health" : 1,
                            "state" : 2,
                            "stateStr" : "SECONDARY",
                            "uptime" : 256214,
                            "optime" : {
                                    "ts" : Timestamp(1584084717, 1),
                                    "t" : NumberLong(2)
                            },
                            "optimeDurable" : {
                                    "ts" : Timestamp(1584084717, 1),
                                    "t" : NumberLong(2)
                            },
                            "optimeDate" : ISODate("2020-03-13T07:31:57Z"),
                            "optimeDurableDate" : ISODate("2020-03-13T07:31:57Z"),
                            "lastHeartbeat" : ISODate("2020-03-16T06:53:22.760Z"),
                            "lastHeartbeatRecv" : ISODate("2020-03-16T06:53:22.777Z"
    ),
                            "pingMs" : NumberLong(0),
                            "syncingTo" : "xxxx:27018",
                            "configVersion" : 104289
                    },
                    {
                            "_id" : 1,
                            "name" : "xxxx:27018",
                            "health" : 1,
                            "state" : 1,
                            "stateStr" : "PRIMARY",
                            "uptime" : 256217,
                            "optime" : {
                                    "ts" : Timestamp(1584341601, 1),
                                    "t" : NumberLong(57)
                            },
                            "optimeDate" : ISODate("2020-03-16T06:53:21Z"),
                            "electionTime" : Timestamp(1584086043, 1),
                            "electionDate" : ISODate("2020-03-13T07:54:03Z"),
                            "configVersion" : 104289,
                            "self" : true
                    }
            ],
            "ok" : 1
    }

I would strongly recommend upgrading to the latest (and final) version of MongoDB 3.4 (3.4.24). There have been a significant number of bug fixes and stability improvements in the 3.4 release series and upgrading will confirm you are not encountering a known issue since 3.4.5 was released back in June, 2017. Minor releases do not introduce any backward-breaking changes.

i would also note that MongoDB 3.4 reached end of life in January, 2020 so I would also plan to upgrade to 3.6 for continued stability & security improvements.

I was referring to the roles of replica set members (primary, secondary, arbiter). However, that isn’t relevant since you have provided the rs.status() info.

A lastCommittedOpTime of -1 is expected if you are running a replica set with the older replication protocol 0 (pv0).

You can confirm your protocol version using rs.conf().protocolVersion.

Regards,
Stennie

1 Like

Thanks Stennie.

Protocol version:
myShard_0:PRIMARY> rs.conf().protocolVersion
NumberLong(1)
myShard_0:PRIMARY>

Hrm, in this case the lastCommittedOpTime should normally have a t value.

Can you also share the output of rs.conf() from your primary with the hostnames redacted?

In your original post you mentioned replication was going fine previously. Did anything specific change in your deployment (configuration, upgrades, election, …) to prompt noticing this change in the replication status output?

Thanks,
Stennie

1 Like

Thanks Stennie. I just did role reversal. Primary to secondary and vice versa.

myShard_0:PRIMARY> rs.conf()
{
        "_id" : "myShard_0",
        "version" : 104289,
        "protocolVersion" : NumberLong(1),
        "members" : [
                {
                        "_id" : 0,
                        "host" : "xxx:27017",
                        "arbiterOnly" : false,
                        "buildIndexes" : true,
                        "hidden" : false,
                        "priority" : 1,
                        "tags" : {

                        },
                        "slaveDelay" : NumberLong(0),
                        "votes" : 1
                },
                {
                        "_id" : 1,
                        "host" : "cccwin:27018",
                        "arbiterOnly" : false,
                        "buildIndexes" : true,
                        "hidden" : false,
                        "priority" : 0.5,
                        "tags" : {

                        },
                        "slaveDelay" : NumberLong(0),
                        "votes" : 1
                }
        ],
        "settings" : {
                "chainingAllowed" : true,
                "heartbeatIntervalMillis" : 2000,
                "heartbeatTimeoutSecs" : 10,
                "electionTimeoutMillis" : 10000,
                "catchUpTimeoutMillis" : 2000,
                "getLastErrorModes" : {

                },
                "getLastErrorDefaults" : {
                        "w" : 1,
                        "wtimeout" : 0
                },
                "replicaSetId" : ObjectId("5e6b3245ccc189cd553cf737")
        }
}
myShard_0:PRIMARY>

How can I resolve this @Stennie_X. Please help.

Thanks,
Akshaya Srinivasan

Hi,

I am seeing this issue again. i am using MongoDB version 3.6.20 , OS is RHEL 6.7 . Please can some one help? Also protocolVersion is not listed in the rs.conf() output.

Secondary node:

[root@akscent1 Log_Files]# /root/mongodb-linux-x86_64-rhel62-3.6.20/bin/mongo --port 27097 -u xxxx -p’xxxx’ --authenticationDatabase admin --eval “rs.status().optimes.lastCommittedOpTime”

MongoDB shell version v3.6.20

{ “ts” : Timestamp(0, 0), “t” : NumberLong(-1) }

Secondary node:

[root@akscent1 Log_Files]# /root/mongodb-linux-x86_64-rhel62-3.6.20/bin/mongo --port 27098 -u xxxx -p’xxxx’ --authenticationDatabase admin --eval “rs.status().optimes.lastCommittedOpTime”

MongoDB shell version v3.6.20

{ “ts” : Timestamp(0, 0), “t” : NumberLong(-1) }

Primary node:

[root@akscent1 Log_Files]# /root/mongodb-linux-x86_64-rhel62-3.6.20/bin/mongo --port 27099 -u xxxx -p’xxxxx’ --authenticationDatabase admin --eval “rs.status().optimes.lastCommittedOpTime”

MongoDB shell version v3.6.20

{ “ts” : Timestamp(1607574936, 1), “t” : NumberLong(-1) }

Thanks,
Akshaya Srinivasan

Hi @Akshaya_Srinivasan

What’s the whole output of rs.status() and rs.conf()? Also please show the output of rs.printReplicationInfo().

Best regards,
Kevin