I’m using mongo 4.2, trying to perform an update using aggregation pipeline. Mongo transaction management is in place to roll back the updates for failure scenarios.
For Mongo transactions, it is recommended to limit the number of documents to 1000 and for documents greater than the limit, developer is expected to split the document in batches.
In my case, I run an update using a generic field. Example: Update all documents with "TaskId" : 1234 with a new field "Status": "Processed".
There can be “N” documents with TaskId as 1234, at times below 1000 and at times greater than 1000. In such case, is it recommended to use mongo transactions, If yes, how can I split them as batches and run an update within transaction management.
I notice you haven’t had a response to this topic yet - were you able to find a solution?
If not, could you confirm if my understanding for your use-case is correct?
I think you are referring to this post of best practices for transactions in MongoDB. This article explains the recommended best practices for transactions. However, please note that those are separate from the actual operational limitations of transactions as described in this documentation page of operational limits of Multi-document Transactions. Note that best practices may change from version to version due to improvements in MongoDB. However, keeping document modifications less than 1000 per transaction is still recommended, as far as I know. Having said that, have you tried running transactions in which more than 1000 document modifications were involved and did you see any errors/issues?
Coming back to your use case, I understand that sometimes if TaskId cannot be split into less than 1000 documents, are there any other fields that could be used in conjunction with TaskId (by acting as a secondary key)? For example: date, other ID sequences, names, etc. so that you can always split a transaction batch in a sub-TaskId level?
If you need more information, please provide an example document.