Failed to send "getMore" command with database "<DBName>": Failed to read 16716027 bytes: socket error or timeout: generic server error

Working with Data Drivers
1

Hi,

I use Mongocxx to query the single mongod running on linux.
I have an aggregation query which might return > 1 documents on the cursor but I am getting the exception - “Failed to send “getMore” command with database “”: Failed to read 16716027 bytes: socket error or timeout: generic server error”, I believe this is due to cursor getting timedout before even I process the current document.
Actually while iterating over the cursor, for each document I have a gridfs file ID saved so I get the file_id from the document and then get the data from gridfs using gridfs bucket apis ( open_download_stream and read ), can this take time enough to result in cursor timeout ? If yes then is there a way to keep the server alive with any query flag while I process the gridfs api?

MaxTimeMs is the one which is for getting the data from dbserver, in this case I guess the data got retrieved within the timeout ?
noCursorTimeout is the option only available with find query, not with aggregate.

Could someone tell which query option I can use to avoid this kind of errors ?

2

By default, MongoDB closes a cursor when the client has exhausted all the results in the cursor. Which might be happening here as you’re downloading files from GridFS.

Here are a few strategies you can consider:

  1. Use batchSize in your aggregation query: Setting a smaller batch size might help by reducing the amount of data loaded into memory at once, thereby reducing the total processing time for each batch of documents.
  2. Reduce processing time per document: If possible, optimize the process of reading from GridFS to make it faster. You might consider processing documents in parallel or using threads to download files from GridFS while you continue iterating over the cursor.
  3. Manually keep the cursor alive: Although there isn’t a direct way to use noCursorTimeout with an aggregation operation, you could implement a method to keep the cursor “alive” by periodically sending small read operations to the database. However, this can be complex to manage and may not be the cleanest solution.
  4. Adjust socketTimeoutMS and maxTimeMS on the client-side: Ensure that the socket timeout is configured appropriately on the client side to accommodate the time spent fetching data from GridFS. You may want to increase this time to ensure there’s no timeout before your operations complete. The maxTimeMS usually refers to the maximum time the server allows for executing the operation, so also ensure this isn’t causing the issue.