MongoDB MongoS Connection Timeout in Sharded Cluster

Dear Support,

We’re having a strange problem. We are currently running a 3-node MongoDB shared cluster. Once in a while, the connection timeouts to MongoS occur on all the nodes in the cluster. To resolve the issue, we are performing a manual failover from a current primary node in the Mongo config cluster. Following the failover, Mongodb is resumed accepting new connections to mongos. We investigated the logs in the MongoS/Config/Shard node and found no odd error messages, only connection disconnection problems in the cluster. I have provided some logs for reference. Can somebody assist with this problem?

MongoDB Version: 4.2.8

Config Server Log:
2023-12-13T09:08:06.622+0000 I NETWORK [conn18304] end connection 192.X.X.X:33632 (61 connections now open)
2023-12-13T09:11:22.911+0000 I SHARDING [PeriodicShardedIndexConsistencyChecker] Checking consistency of sharded collection indexes across the cluster
2023-12-13T09:11:22.921+0000 I SHARDING [PeriodicShardedIndexConsistencyChecker] Found 0 collections with inconsistent indexes
2023-12-13T09:11:46.868+0000 I NETWORK [conn2747] end connection 192.X.X.X:35292 (60 connections now open)
2023-12-13T09:11:47.377+0000 I NETWORK [conn18021] end connection 192.X.X.X:37720 (59 connections now open)
2023-12-13T09:11:50.872+0000 I NETWORK [conn18235] end connection 192.X.X.X:59180 (58 connections now open)
2023-12-13T09:21:22.911+0000 I SHARDING [PeriodicShardedIndexConsistencyChecker] Checking consistency of sharded collection indexes across the cluster

MongoS Logs:

2023-12-13T09:13:11.508+0000 I NETWORK [conn4030793] end connection 192.X.X.X:38640 (228 connections now open)
2023-12-13T09:13:12.330+0000 I NETWORK [listener] connection accepted from 192.X.X.X:58440 #4030888 (229 connections now open)
2023-12-13T09:13:12.330+0000 I NETWORK [conn4030888] received client metadata from 192.X.X.X:58440 conn4030888: { driver: { name: “mong
o-go-driver”, version: “v1.1.1” }, os: { type: “linux”, architecture: “amd64” }, platform: “go1.11.13” }
2023-12-13T09:13:12.349+0000 I NETWORK [listener] connection accepted from 192.X.X.X:41556 #4030889 (230 connections now open)
2023-12-13T09:13:12.349+0000 I NETWORK [conn4030889] received client metadata from 192.X.X.X:41556 conn4030889: { driver: { name: “mong
o-go-driver”, version: “v1.9.1” }, os: { type: “linux”, architecture: “amd64” }, platform: “go1.18.2” }
2023-12-13T09:13:12.352+0000 I - [conn4030870] operation was interrupted because a client disconnected
2023-12-13T09:13:12.352+0000 I NETWORK [conn4030870] DBException handling request, closing client connection: ClientDisconnect: operation w
as interrupted
2023-12-13T09:13:12.352+0000 I NETWORK [conn4030870] end connection 192.X.X.X:58326 (229 connections now open)
2023-12-13T09:13:12.369+0000 I NETWORK [listener] connection accepted from 192.X.X.X:41564 #4030890 (230 connections now open)
2023-12-13T09:13:12.369+0000 I NETWORK [conn4030890] received client metadata from 192.X.X.X:41564 conn4030890: { driver: { name: “mong
o-go-driver”, version: “v1.9.1” }, os: { type: “linux”, architecture: “amd64” }, platform: “go1.18.2” }
2023-12-13T09:13:12.404+0000 I - [conn4030796] operation was interrupted because a client disconnected
2023-12-13T09:13:12.405+0000 I NETWORK [conn4030796] DBException handling request, closing client connection: ClientDisconnect: operation w
as interrupted
2023-12-13T09:13:12.405+0000 I NETWORK [conn4030796] end connection 192.X.X.X:41272 (229 connections now open)
2023-12-13T09:13:12.419+0000 I - [conn4030797] operation was interrupted because a client disconnected
2023-12-13T09:13:12.419+0000 I NETWORK [conn4030797] DBException handling request, closing client connection: ClientDisconnect: operation was interrupted
2023-12-13T09:13:12.419+0000 I NETWORK [conn4030797] end connection 192.X.X.X:41282 (228 connections now open)
2023-12-13T09:13:12.531+0000 I NETWORK [listener] connection accepted from 192.X.X.X:47440 #4030891 (229 connections now open)
2023-12-13T09:13:12.531+0000 I NETWORK [conn4030891] received client metadata from 192.X.X.X:47440 conn4030891: { driver: { name: “mongo-go-driver”, version: “v1.4.1” }, os: { type: “linux”, architecture: “amd64” }, platform: “go1.18.2” }
2023-12-13T09:13:12.534+0000 I NETWORK [listener] connection accepted from 192.X.X.X:47446 #4030892 (230 connections now open)
2023-12-13T09:13:12.534+0000 I NETWORK [conn4030892] received client metadata from 192.X.X.X:47446 conn4030892: { driver: { name: “mongo-go-driver”, version: “v1.4.1” }, os: { type: “linux”, architecture: “amd64” }, platform: “go1.18.2” }
2023-12-13T09:13:12.574+0000 I - [conn4030799] operation was interrupted because a client disconnected
2023-12-13T09:13:12.574+0000 I NETWORK [conn4030799] DBException handling request, closing client connection: ClientDisconnect: operation was interrupted

Best Regards,
Ashwin