Mongoose with mongo atlas M10 suddenly becomes drastically slow

I have been using Mongoose Driver for my node js application, everything works fine but suddenly queries become damn slow after certain hours or days.

Earlier my queries were taking around 20-40 ms to respond back to the client, now the same queries take 30-40 seconds using the same configuration and internet bandwidth and no code changes.

If restart the application server again, queries start running faster again.

As per my research, it must be due to the connection pool of mongoose, I am using the default
configuration of maxpoolsize 100, minpoolsize 0, (as per the underlying mongo db driver document) along with the following options

useNewUrlParser: true,
useUnifiedTopology: true,
autoIndex:false,family: 4’

Can you please help me ? Will the minPoolSize solve the problem?