We have a standalone MongoDB server and for some reason we are seeing that Mongodb is no longer accepting any new connections the weird part is that
I’m attempting to make a Mongodb connection
mongo ‘mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb&socketTimeoutMS=100000&connectTimeoutMS=100000’
Received Timeout
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&connectTimeoutMS=100000&gssapiServiceName=mongodb&socketTimeoutMS=100000
Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection timed out :
connect@src/mongo/shell/mongo.js:374:17
@(connect):2:6
exception: connect failed
exiting with code 1
I see the following result
but nothing is visible on the mongodb.log
The only thing is visible are …
iredTiger message","attr":{"message":"[1688542311:637059][10933:0x7f3a7403f700], WT_SESSION.checkpoint: [WT_VERB_CHECKPOINT_PROGRESS] saving checkpoint snapshot min: 341775, snapshot max: 341827 snapshot count: 3, oldest timestamp: (0, 0) , meta checkpoint timestamp: (0, 0)"}}
{"t":{"$date":"2023-07-05T07:32:02.744+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"127.0.0.1:46572","connectionId":19534,"connectionCount":77}}
{"t":{"$date":"2023-07-05T07:40:00.570+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"127.0.0.1:56164","connectionId":19535,"connectionCount":78}}
{"t":{"$date":"2023-07-05T08:40:17.003+00:00"},"s":"I", "c":"-", "id":20883, "ctx":"conn20","msg":"Interrupted operation as its client disconnected","attr":{"opId":1854086}}
{"t":{"$date":"2023-07-05T08:40:17.009+00:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn20","msg":"Connection ended","attr":{"remote":"127.0.0.1:58894","connectionId":20,"connectionCount":78}}
I can see that Connection Count is 78 but I don’t know the max value for it.
I suspect this could be because of Max Open Files but I doubt it because of the following
a) No such error is visible on the logs
b) The clients are timing out instead of returning failure immediately.
c) the systemd unit has the value set to 64000
I have a feeling if I restart the server everything will become normal but we would like to know
we can prevent such problem in future.
And I was right after the restart things did work 
Also, I’m not sure what the max open files limit for Mongodb user from systemclt unit it seems like 64000
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false
# mongod.conf
storage:
dbPath: /var/lib/mongodb
journal:
enabled: true
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
net:
port: 27017
bindIp: 127.0.0.1
processManagement:
timeZoneInfo: /usr/share/zoneinfo
Mongo db version we have
db version v4.4.5
Build Info: {
"version": "4.4.5",
"gitVersion": "ff5cb77101b052fa02da43b8538093486cf9b3f7",
"openSSLVersion": "OpenSSL 1.1.1n 15 Mar 2022",
"modules": [],
"allocator": "tcmalloc",
"environment": {
"distmod": "debian10",
"distarch": "x86_64",
"target_arch": "x86_64"
}
}
Mongo shell version
MongoDB shell version v4.4.5
Build Info: {
"version": "4.4.5",
"gitVersion": "ff5cb77101b052fa02da43b8538093486cf9b3f7",
"openSSLVersion": "OpenSSL 1.1.1n 15 Mar 2022",
"modules": [],
"allocator": "tcmalloc",
"environment": {
"distmod": "debian10",
"distarch": "x86_64",
"target_arch": "x86_64"
}
}
I can confirm with systemd unit file the File limit is set to 640000