The number of current connections in a mongodb replica is not reducing (stuck at some higher value 900) until the mongo replica restarts

Other MongoDB Topics Open Source Projects
1

I have deployed Mongodb (v4.4.2) replicaset on a kubernetes platform with 1 PRIMARY and 2 SECONDARY replicas.

There are a lot of READ/WRITE connections that are coming to mongodb from various applications. Due to this the current connections are touching a huge number like 900+.

The mongodb connections are stuck at 900+ for long hours and never comes back to its original value.

Because of this the PRIMARY mongodb replica is keep on restarting thus blocking the applications from writing to mongodb.

Number of connections in Idle state:

rs0:PRIMARY> db.serverStatus().connections
{
 "current" : 51,
 "available" : 838809,
 "totalCreated" : 7670,
 "active" : 5,
 "exhaustIsMaster" : 2,
 "exhaustHello" : 1,
 "awaitingTopologyChanges" : 6
}

Number of connections after inserting data from apps:

rs0:PRIMARY> db.serverStatus().connections
{
 "current" : 898,
 "available" : 837962,
 "totalCreated" : 3396,
 "active" : 4,
 "exhaustIsMaster" : 1,
 "exhaustHello" : 1,
 "awaitingTopologyChanges" : 4
}

This use case is really making the Mongodb setup unstable, so we want to make the mongodb stable by reducing the existing connections.

NOTE: We have a restriction on the maximum number of threads being created in an instance i,.e 1024.

Any suggestions would be appreciated.