Connect timeout and execution timeout in nodejs driver

Hi Guys! I hope I’m posting this in correct place…

I’m working with MongoDB using NodeJS mongodb driver. But I can’t make few options to work… My setup:
OS: Windows 10 x64
NodeJS mongodb driver: 3.5.5
MongoDB: 4.2.5

I want to limit:

  • Connection time into DB (connectTimeoutMS option)
  • And query execution time (socketTimeoutMS option)

Description of connectTimeoutMS from mongodb.github.io: How long to wait for a connection to be established before timing out

Description from jira.mongodb.org: dictates how long to wait for an initial connection before considering it a timeout. This is used exclusively when we create a TCP/TLS socket

So I tried to connect to disabled DB with the following code:

const { MongoClient } = require('mongodb')

const url = 'mongodb://localhost'
const config = {
  connectTimeoutMS: 5000,
  useUnifiedTopology: true
}

async function mongodb() {
  console.time('connect')
  try { await MongoClient.connect(url, config) }
  catch (error) { console.error(error) }
  finally { console.timeEnd('connect') }
}

mongodb()

Which produces the following logs:

MongoServerSelectionError: connect ECONNREFUSED 127.0.0.1:27017
connect: 30034.340ms

If I will change useUnifiedTopology to false I will get this log:

(node:19040) DeprecationWarning: current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.

MongoNetworkError: failed to connect to server [localhost:27017] on first connect [Error: connect ECONNREFUSED 127.0.0.1:27017

connect: 2025.937ms

The issue that I have connectTimeoutMS set to 5 000 ms (5 secs) but as you can see it gets timed out in 2 or 30 seconds not in 5 seconds.


Next issue is with query execution timeout.

Description of socketTimeoutMS from mongodb.github.io: How long a send or receive on a socket can take before timing out

Description from jira.mongodb.org: dictates how long to wait for any operation on an existing connection before timing out. Once the TCP socket has been connected, we use this for our actual operations

So I tried to make a query into DB with 5 seconds timeout limit:

const { MongoClient } = require('mongodb')

const fetchLimit = 500000
const url = 'mongodb://localhost'
const config = {
  connectTimeoutMS: 5000,
  socketTimeoutMS: 5000,
  useUnifiedTopology: true
}

async function mongodb() {
  let client

  console.time('connect')
  try { client = await MongoClient.connect(url, config) }
  catch (error) { console.error(error) }
  finally { console.timeEnd('connect') }

  const coll = client.db('reps').collection('req')

  console.time('find')
  try { await coll.find({ p: 'XYZ' }).limit(fetchLimit).toArray() }
  catch (error) { console.error(error) }
  finally { console.timeEnd('find') }
}

mongodb()

And I got this:

connect: 28.459ms
find: 8731.263ms

I though that after 5 secs I will get timeout error… What’s wrong ?
Even if I will set socketTimeoutMS to 30 ms I will get the same result…
But if I will set it to 20 ms I will get this:

connect: 70.304ms
MongoNetworkError: connection 2 to 127.0.0.1:27017 timed out
find: 1225.078ms

If I replace find with count (collection has more than 10 millions docs) and use await coll.countDocuments() I will get Timeout Error after 60 secs (not after 5)

Can anybody assist me, I dont understand what’s going on…

2 Likes

Hi @Vlad_Kote, welcome to the community forums! I’ll try to answer your questions in two sections below:

connectTimeoutMS

The connectTimeoutMS option is used by the driver to determine when to timeout an attempt to connect an individual connection to a server (one of many in your connection pool). It does not have a direct relation to MongoClient#connect. When you attempt to connect a MongoClient it attempts server selection, which means that it will attempt for up to serverSelectionTimeoutMS to connect to the cluster before reporting that it was unable to find a suitable server.

If you want to get “fast fail” behavior on your connect, you can pass serverSelectionTimeoutMS to your MongoClient:

const client = new MongoClient(..., { serverSelectionTimeoutMS: 5000 });
await client.connect();

socketTimeoutMS

The socketTimeoutMS option corresponds to Node’s Socket#setTimeout method, and guides the behavior around socket inactivity. In your case it seems like you want to guarantee that an operation succeeds or fails in a given time range. Your best bet for this today is to use a combination of socketTimeoutMS (in case the socket is indeed inactive, due to network issues) and maxTimeMS which will cause the operation to fail after the specified time on the server side:

try {
  const collection = client.db().collection('test_collection');

  console.time('find');
  await collection.find({ $where: 'sleep(1000)' }).limit(10).maxTimeMS(10).toArray();
  console.timeEnd('find');
} catch (err) {
  console.timeEnd('find');
  console.dir({ err });
}

in this case will result in:

find: 44.282ms
{
  err: MongoError: operation exceeded time limit
      at MessageStream.messageHandler (/home/mbroadst/Development/mongo/node-mongodb-native/lib/cmap/connection.js:261:20)
      at MessageStream.emit (events.js:209:13)
      at processIncomingData (/home/mbroadst/Development/mongo/node-mongodb-native/lib/cmap/message_stream.js:144:12)
      at MessageStream._write (/home/mbroadst/Development/mongo/node-mongodb-native/lib/cmap/message_stream.js:42:5)
      at doWrite (_stream_writable.js:428:12)
      at writeOrBuffer (_stream_writable.js:412:5)
      at MessageStream.Writable.write (_stream_writable.js:302:11)
      at Socket.ondata (_stream_readable.js:722:22)
      at Socket.emit (events.js:209:13)
      at addChunk (_stream_readable.js:305:12) {
    ok: 0,
    errmsg: 'operation exceeded time limit',
    code: 50,
    codeName: 'ExceededTimeLimit',
    operationTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_: 1585574598 },
    '$clusterTime': { clusterTime: [Timestamp], signature: [Object] },
    name: 'MongoError',
    [Symbol(mongoErrorContextSymbol)]: {}
  }
}

Hope that helps!

2 Likes

Thanks a lot, @mbroadst !

And yes, ‘serverSelectionTimeoutMS’ exactly what I need.
And ‘maxTimeMS’ method also did the trick here…

Looks like I was confused from the beginning…
I’m working with Mongoose lib which has clear notice in the docs that “we pass Option object into underlying mongodb lib”. But I didn’t find such notice in mongodb lib. That’s why I thought that mongodb couldn’t handle it…

So serverSelectionTimeoutMS option is handled by… By whom ? The mongodb lib source uses that param but I can’t find mention about it in their docs mongodb.github.io/node-mongodb-native/3.5/api/. Am I looking in wrong place ?

Lib docs hasn’t Connection Settings

But the MongoDB docs itself has…

I will try now to achieve what I want with Mongoose.

Again, thanks a lot, @mbroadst

Glad to hear it helped!

You’re not crazy, our documentation is not helping you out right now - but we’re working on that:

  • serverSelectionTimeoutMS is a top-level MongoClient or connection string option that is only supported by the “Unified Topology”. Since this topology is still gated by a feature flag, its documentation was not merged in with MongoClient. In the next patch release of the driver these options will be added, with a note that they are only relevant to the unified topology.

  • We are currently introducing type checking to the driver, which will result in much higher quality API documentation, and intellisense if you happen to be using an editor with LSP support

  • Finally, we have a larger project in place to rewrite our Reference documentation to include in-depth details on topics such as this. The project is being handled by our fantastic documentation team, so you can expect to have a better experience with our documentation in the future.

3 Likes

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.