Mongotop fails after sometime with error Failed: BSONObj size: 59797193 (0x3906EC9) is invalid. Size must be between 0 and 16793600(16MB)

Ops and Admin
1

Hi,

When server is started, mongotop works. After a few hours/days, mongotop fails. The error shown is of the form:
2022-07-01T01:38:14.747+0000 Failed: BSONObj size: 59797193 (0x3906EC9) is invalid. Size must be between 0 and 16793600(16MB) First element: note: “all times in microseconds”

If the mongod service is restarted, mongotop works again. But, again after a few hours/days, mongotop fails again with the same error above.

Another case is, when a script is run to check document object size across all documents in all collections in all databases, script fails after a while. It is Never always the same database/collection/document.

Please help in understanding why mongotop is failing randomly and make it work reliably without any failure.

Thank you,
Melvin

3

Hi @Melvin_George,

Welcome to the MongoDB Community forums again :sparkles:

This error is raising because it’s hitting more than the max size of BSONObj i.e., 16 MB and the mongotop command currently returns its output as a single BSON document so the result is limited to 16MB and here it’s 60MB.

So, restarting the mongod clears the in-memory information and it works again!

However, to better understand the issue could you please provide us a few details:

  • What version of MongoDB you are using?
  • How many active collections do you have?
  • What information you are looking for in mongotop?

Best,
Kushagra

1 Like
4

Thanks for replying, Kesav.

To your questions, please find answers below:

  • What version of MongoDB you are using?
    MongoDB 4.4.14 (same error happened with 4.4.6, so we upgraded to 4.4.14 and the error still exists)

  • How many active collections do you have?
    Around 80 databases, with mostly 1300 collections each.

  • What information you are looking for in mongotop?
    Identify the database.collection having most activity during peak load.

I suppose you are trying to understanding whether we are going past any limits configured by default. Appreciate your help.

Regards,
Melvin

1 Like
5

Hi @Melvin_George,

Thanks for sharing these details!

Unfortunately, I think you’re hitting the issue SERVER-6627, where in a very large deployment, the output of the top command (which mongotop relies on) can get past the 16MB BSON size limitation. I would encourage you to comment & upvote on the ticket to help the development team prioritize this issue.

In the meantime, I think you can use the output of db.currentOp() as a workaround.

Please let us know if you have any follow-up questions!

Kind Regards,
Kushagra

1 Like
7

Thanks for your response, Kushagra.

Have commented and upvoted the SERVER-6627 bug.

Will check db.currentOp() and get back on this.

Regards,
Melvin

8

Hi Kushagra,

I looked at:

use admin
db.currentOp()

Seems it also has a limit of 16MB on the final document size:

But, thanks to your hint above, I came across $currentOp aggregation stage:

use admin

db.aggregate([{$currentOp: {allUsers: true, idleConnections: true}}, {$match: {$and: [{"ns": {$exists: true}}, {"ns": {$nin: ["", "admin.$cmd", "admin.$cmd.aggregate"]}}]}}, {$project: {ns: 1, microsecs_running: 1, op: 1}}]).toArray()

It returns a cursor that can build a document with details found in db.currentOp(). The cursor goes over documents each of which should not be greater than 16MB. But, the aggregate will build a document that has No size restrictions. Can give all info in mongotop, plus more.

Thanks, Kushagra!

1 Like
9

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.