MongoDB insertMany stalls at callback

Working with Data
,
1 
async function insertManyCallback(collectionName, query, options, callback) {
  
  try {
await MongoClient.connect(
  dburl,
  defaultOptions,
  function(err, client) {
    if (err) logger.error(err);

    if (!err) {
      client
        .db(dbName)
        .collection(collectionName)
        .insertMany(query, options, function(err, res) => {
          client.close();
          callback();
        });
    }
  }
);

  } catch (err) {
logger.error(err);
  }

  return true;
}

I’m inserting approximately 65K documents for each bulk write operation, and I understand it will be slow. My problem stems from me querying the MongoDB collection for count documents. I can see the number go up and up until it has “visibly” finished inserting all the documents. Then, the insertMany just freezes there for a minute without any visible changes to the collection before calling my callback function.

Does anyone know why it freezes for a minute and anything I can do about it?

I also tried .then instead of function(err, res) to try and bypass it returning a result; the same outcome in terms of performance.

2

Hi @Dogunbound_hounds,

I believe this requires a server side performance investigation as it might be that the MongoDB cluster is overwhelmed with the write workloads and stalls to reclaim resources.

What is the monogo version of server and driver?

Have you tried adding resources to the cluster or monitoring their utilisation?

I recommend reading this article and its reference

What is the writeConcern you are using? Try using w majority if this is a replica set…

To set the expectations here, I am not sure how community can deeply investigate server performance issues and I strongly recommend case contacting support who specialise in those areas…

Thanks
Pavel

3

So this is a server performance issue. I was just trying to pinpoint what I was doing wrong. It isn’t a replica set, nor can I edit writeConcerns because the database is locally running with the backend.

Thanks for telling me this is purely a performance issue.

4

@Dogunbound_hounds,

Without looking into the specific environment details and diagnostics I cannot tell anything, however, usually those are related to performance issues…

Thanks,
Pavel