Concurrent connections to mongodb is drastically slow

I am using lambda (nodeJS) with mongoDB atlas. The lambda creates a new connection to mongoDB every time it is invoked. It works perfectly fine with fewer requests. However, if I make 100 concurrent requests, assuming 100 new instance of lambda is created, 100 concurrent connections to mongoDB atlas is initiated. The connection time is then keep on increasing starting from 100ms to as high as 30 seconds.

I am using M10 cluster tier of mongoDB which supports 1500 concurrent connections and I am using following code block for connection:

if (cachedDB) {
        try {
            cachedDB = await mongoose.connect(uriString, {
                useUnifiedTopology: false,
                useNewUrlParser: true,
                useFindAndModify: false,
                poolSize: 3,
            });
        } catch (err) {
            logger.error(err);
            cachedDB = null;
        }
}

I am not sure what’s wrong here. Can’t we have concurrent mongoDB connections?

Hi @Pratik_Raj2 welcome to the community!

AWS Lambda has always been tricky to manage in terms of database connection, due to the nature of the service itself. That is, as far as I understand, AWS Lambda guarantees the execution of your function, but not the environment or other operational concerns regarding that execution. It is geared toward stateless functions, and MongoDB drivers by default works in a stateful manner.

Please have a look at Manage Connections with AWS Lambda for best practices regarding connecting from AWS Lambda.

However in my opinion, the cleanest way to do this is to create an API layer in front of the MongoDB server. This can be deployed separately, or you can use a custom HTTPS endpoints in Atlas. This layer will be responsible for connection to MongoDB, and the Lambda function will be calling this endpoint for database access. Since it’s only the API layer that connects to the database, you can easily monitor its performance and you won’t have this concurrent connection issues that takes a lot of effort to solve. You can also more easily secure the database since you only need to give IP access to the API layer.

Best regards
Kevin

1 Like