Reproducing CursorNotFound error due to cursor inactivity and session idle timeout

[shrey]

How do I reproduce CursorNotFound error due to 10 minutes of cursor inactivity and due to session idle timeout in 30 minutes using PyMongo tools?

I am trying something like this :

def sleep_for_minutes(minutes_to_sleep):
  for i in range(minutes_to_sleep):
    print(f'{i} sleeping for 1 minute')
    time.sleep(60 * 1)
    

# Iterate over all documents in the collection
for document in collection.find():
  print(f'{document} before sleeping')

  sleep_for_minutes(15)
  # even tried sleeping for 35 minutes but didn't help
  # sleeping for 45 mins worked (don't know why)

  print(f'{document} after sleeping')

Context

Am iterating a very large Mongo collection. Each document of a collection is quite large as well. Tech stack : MongoEngine, Django

My production system is timing out due to CursorNotFound error.

Error goes like this : pymongo.errors.CursorNotFound: cursor id <something> not found, full error: {'ok': 0.0, 'errmsg': 'cursor id <something> not found', 'code': 43, 'codeName': 'CursorNotFound'}

As per my understanding, there can 2 possible reasons for the same:

• Session Idle Timeout which occurs at 30 minutes
• Cursor inactivity timeout after 10 minutes of inactivity

To fix and verify the fixes, I am trying to reproduce there errors on a local setup to fix the issue. I do this by using sleep methods.

[Shane (Shane Harvey)]

The MongoDB server task that actually times out and expires the session only runs every 5 minutes by default: https://www.mongodb.com/docs/manual/reference/parameters/#mongodb-parameter-param.logicalSessionRefreshMillis. This helps explain why 35 minutes of inactivity did not reproduce the timeout but 45 minutes did (the task was probably running or about to run). To reproduce the issue you can sleep for 37+ minutes or decrease the logicalSessionRefreshMillis and localLogicalSessionTimeoutMinutes.

[shrey]

Thanks for the response Shane.
I was able to reproduce the session idle timeout flow based on what you suggested.

BUT, I am still not able to reproduce the cursor timeout (10 mins default) due to inactivity.

Following are my test steps:

Setting params in mongo

docker run -d -p 27017:27017 --name mongodb-container mongo --setParameter cursorTimeoutMillis=10000 --setParameter logicalSessionRefreshMillis=10000

Making the actual query

# sleep is to emulate slow query
cursor = collection.find(batch_size=1).where("sleep(59000) || true")

for document in cursor:
  print(f'sleeping and processing document: {document}')
  sleep_for_minutes(3)
  print(f'sleeping done')

This works without erroring out as such.

What exactly is the meaning of “cursor inactivity” or “cursor being idle”?
I guess I might have got it wrong.

[Shane (Shane Harvey)]

In 4.4.8+ MongoDB always sets the noCursorTimeout option for cursors that are created with a session: https://jira.mongodb.org/browse/SERVER-6036

So there is no more 10 minute cursor timeout, only the 30 minute session timeout. When the session is expired the cursor(s) it created are closed as well.