Too many connections using node driver with atlas

Will_Smith1 · September 5, 2023, 6:59pm

I’ve opened an issue on the mongo remix example here.

Too many connections?

opened 06:59PM - 30 Jul 23 UTC

I've got a remix app with the same setup as in this repo. In each request I do t…his: ``` let db = await mongodb.db("sample_mflix"); let collection = await db.collection("movies"); ... ``` And that's all setup in my db.ts file ``` if(process.env.NODE_ENV === "production") { mongodb = new MongoClient(connectionString); } else { if(!global.__db) { global.__db = new MongoClient(connectionString); } mongodb = global.__db; } ``` Functionally it's all fine and working well, but I'm seeing a lot of connections on my production atlas nodes. You can see in the screenshot the number of connections seems to drop when I deploy. The throughput is still tiny (<1/s/node). <img width="607" alt="image" src="https://github.com/mongodb-developer/remix/assets/26863411/cd8cbb2f-e1bd-42ee-8801-eccfb7eba242"> I'm finding it hard to confirm if I'm using the mongo driver correctly. Should I move these bits also into global singletons? Do I then have to worry about retries if the connections drop? ``` let db = await mongodb.db("sample_mflix"); let collection = await db.collection("movies"); ``` The slowly increasing simultaneous connections really worry me as we scale up soon. Thanks a lot!

After not hearing back on the issue, I contacted support who pointed me here.

Is anyone able to confirm if I’m using the driver correctly? It’s a serious concern for me that the connections increase until a new deployment.

I can’t find a definitive answer on whether I should be storing the client in memory, like I am, storing the “db” in memory, or even storing each “collection” in memory.

In contrast to the remix example, the mongo university example stores the “collection” in memory.

github.com

mongodb-university/mflix-js/blob/master/src/dao/moviesDAO.js

import { ObjectId } from "bson"

let movies
let mflix
const DEFAULT_SORT = [["tomatoes.viewer.numReviews", -1]]

export default class MoviesDAO {
  static async injectDB(conn) {
    if (movies) {
      return
    }
    try {
      mflix = await conn.db(process.env.MFLIX_NS)
      movies = await conn.db(process.env.MFLIX_NS).collection("movies")
      this.movies = movies // this is only for testing
    } catch (e) {
      console.error(
        `Unable to establish a collection handle in moviesDAO: ${e}`,
      )
    }

This file has been truncated. show original

Any help would be greatly appreciated. Thanks.

Kushagra_Kesav · October 10, 2023, 1:24am

Hey @Will_Smith1,

Welcome to the MongoDB Community!

You can refer to official MongoDB documentation as well as the MongoDB university course - Connecting to MongoDB in Node.js to learn the right approach to utilize the Nodejs driver within your application to connect to MongoDB Atlas.

May I ask what specific issues you are facing with the increased number of connections? Also, could you further clarify “connections increase until a new deployment”?

However, here the drivers maintain a connection pool, i.e., monitoring threads, rather than a single connection for all the operations, so there can be multiple connections to the cluster from a client.

It depends on your application’s use case and your desired approach for connection reuse. If your application frequently interacts with a particular database, it can be beneficial to store it in a variable and access it throughout the application. This concept extends to the client connection as well. By storing and reusing these connections, you can optimize your application’s performance and resource usage. Eventually, the decision should align with your application’s use case and requirements.

For your reference, here is example code snippet of reusing the connection from the StackOverflow response, which might be helpful to you.

Best regards,
Kushagra