Mongodb memory spiked to 100% reporting serverstatus slow and NetworkInterface error

Hi All,

Recently, two secondary nodes failed and 1 primary gave-up the election and resulted in whole cluster being down. it looks like secondary nodes reports that “serverStatus was very slow” and networkinterfaceExceededTimeLimit error. there was no heavy operation other than mongodump was running at its usual time. There were indications of “slow Query” including the mongodump. At the same time, mongo memory just reached 100% before the whole node goes down. the same repeated on other secondary node as well. There is no indication of any network issue.

Is there better way to debug this issue and what caused the memory spike?

Using Mongodb 6.0.5 community edition

"ctx":"ftdc","msg":"serverStatus was very slow"
"ctx":"ReplicaSetMonitor-TaskExecutor","msg":"RSM received error response","attr":{"host":"<removed>","error":"NetworkInterfaceExceededTimeLimit : Request 12 timed out, deadline was <removed>, op was RemoteCommand 12 -- target:[<removed>  db:admin expDate:<removed> cmd:{ isMaster: 1, maxAwaitTimeMS: 10000, topologyVersion: { processId : ObjectId('64983..'), counter: 3 }, internalClient: { minWireVersion: 13, maxWireVersion: 17 } }","