Executing multiple queries at the same time with mongoose makes the last ones very slow

Working with Data Node.js Frameworks
,
1

My queries target two different collections. I execute them 23 times (so 23 * 2 = 46 queries).

They look like:

{
  "areaId": ObjectId("63f882962d65a5368041929e"), // 23 different ids
  "from": {
    "$lte": ISODate("2024-02-25T14:00:00.000Z")
  },
  "to": {
    "$gte": ISODate("2024-02-25T14:00:00.000Z")
  },
  "type": {
    "$in": [
      "abc",
      "def"
    ]
  },
  "result": {
    "$ne": "unavailable"
  }
}

I know I can use a $in to make it in a single query. But for the purpose of testing, I need to do it in 23 different queries. Here is an example of execution time for each query:

collectionA-6512a8ecdbadd710cace2c00: 997.466ms
collectionA-6512a90fdbadd710cace2c13: 1.104s
collectionB-6512a987dbadd710cace2c4e: 1.180s
collectionB-651d7855aa6d4dc89c14c04c: 1.476s
collectionB-651d78f2aa6d4dc89c14c0dc: 1.482s
collectionB-651d799faa6d4dc89c14c113: 1.741s
collectionB-651d7881aa6d4dc89c14c080: 1.788s
collectionB-651d7a07aa6d4dc89c14c1dd: 1.848s
collectionB-651d7fd1aa6d4dc89c14c50f: 1.831s
collectionB-63f8ba3d2d65a5368041b3b8: 2.003s
collectionB-651d78cdaa6d4dc89c14c0a5: 2.018s
collectionB-651d84ceaa6d4dc89c14c6a6: 1.950s
collectionA-651d84b3aa6d4dc89c14c67c: 1.981s
collectionB-651d84b3aa6d4dc89c14c67c: 2.054s
collectionA-651d84f3aa6d4dc89c14c6ce: 2.056s
collectionB-651d7a4faa6d4dc89c14c202: 2.123s
collectionA-651d84ceaa6d4dc89c14c6a6: 2.090s
collectionB-651d850baa6d4dc89c14c6e1: 2.107s
collectionA-651d850baa6d4dc89c14c6e1: 2.141s
collectionA-651d7ffaaa6d4dc89c14c535: 2.224s
collectionB-6512a90fdbadd710cace2c13: 2.336s
collectionA-651d7855aa6d4dc89c14c04c: 2.340s
collectionA-63f8baf22d65a5368041b3cb: 2.412s
collectionB-6512a8ecdbadd710cace2c00: 2.365s
collectionA-651d7fd1aa6d4dc89c14c50f: 2.283s
collectionA-651d78f2aa6d4dc89c14c0dc: 2.330s
collectionA-651d78cdaa6d4dc89c14c0a5: 2.362s
collectionA-651d7881aa6d4dc89c14c080: 2.369s
collectionA-651d799faa6d4dc89c14c113: 2.347s
collectionA-651d7a4faa6d4dc89c14c202: 2.346s
collectionA-651d7a07aa6d4dc89c14c1dd: 2.386s
collectionA-651d7fe6aa6d4dc89c14c522: 2.425s
collectionB-651d84f3aa6d4dc89c14c6ce: 2.402s
collectionA-63f8ba3d2d65a5368041b3b8: 2.652s
collectionB-63f8bc1a2d65a5368041b3f1: 2.770s
collectionB-651d7fe6aa6d4dc89c14c522: 2.684s
collectionA-6512a987dbadd710cace2c4e: 2.856s
collectionA-63f8bc1a2d65a5368041b3f1: 2.924s
collectionB-651d7ffaaa6d4dc89c14c535: 2.807s
collectionB-63f8baf22d65a5368041b3cb: 3.020s
collectionB-63f8bcdf2d65a5368041b417: 3.266s
collectionB-63f8869b2d65a53680419572: 3.389s
collectionA-63f8bcdf2d65a5368041b417: 3.488s
collectionA-63f8869b2d65a53680419572: 3.564s
collectionA-63f8bb872d65a5368041b3de: 3.498s
collectionB-63f8bb872d65a5368041b3de: 3.598s

I would like to know why there is this difference between the first and the last query. My mongoose has a maxPoolSize of 50.

2

Hello @fberrez

I suspect the first query may experience initial overhead as it establishes connections to the MongoDB server. Subsequent queries benefit from reusing existing connections from the connection pool, potentially reducing this overhead.

Suppose you reorder the IDs for execution and still experience prolonged query times for specific IDs. In that case, it suggests that documents associated with those IDs in your collection may contain substantial data, contributing to the delay.