Recently have started facing this issue where we are unable to read a chunk of data from AWS DocumentDB. To set the context, our read looks something like this.
batch_data = self._db.find({"key_one": self._value1, "key_two": self._value2}, self._projector).sort("_id", 1).skip(offset).limit(batch_size)
for document in batch_data:
# some code
The keys that I’m storing with are indexed. While this perfectly works with some environments, this isn’t working in production and fails intermittently with the below stack trace.
Traceback (most recent call last):
for document in batch_data:
File "/usr/local/lib/python3.8/site-packages/pymongo/cursor.py", line 1156, in next
if len(self.__data) or self._refresh():
File "/usr/local/lib/python3.8/site-packages/pymongo/cursor.py", line 1073, in _refresh
self.__send_message(q)
File "/usr/local/lib/python3.8/site-packages/pymongo/cursor.py", line 953, in __send_message
response = client._run_operation_with_response(
File "/usr/local/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1342, in _run_operation_with_response
return self._retryable_read(
File "/usr/local/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1464, in _retryable_read
return func(session, server, sock_info, slave_ok)
File "/usr/local/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1334, in _cmd
return server.run_operation_with_response(
File "/usr/local/lib/python3.8/site-packages/pymongo/server.py", line 136, in run_operation_with_response
_check_command_response(first)
File "/usr/local/lib/python3.8/site-packages/pymongo/helpers.py", line 159, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: Cannot update value", "filename": "batch_processer.py"}
The exception occurs in this line where we are trying to loop through our cursor to process individual records.
for document in batch_data: # Error in this line
Also to note that that we aren’t trying an update or an upsert action here. We are plainly trying to read data in batches. There’s only one stackoverflow issue which relates to our stack trace here.
This has started occurring only since last week and we’re unable to find more resources on the internet to fix this issue. Can anyone help us what is going wrong here?