How to handle multiple Nodejs requests reading, modifying and updating the same document at same time

Working with Data Drivers
1

I have the following code where I read data from collection, perform some arithmetic operations on the data and then update the document. My issue is that when multiple requests come at the same time it causes some data to be lost. How can I avoid this?

    //Read the Document
    const commissionRecord = await CommissionTable.getCommissionRecord(
                publicKey
            );

            // Check if A record Exists or Not

            if (commissionRecord.responseData.exists === true) {

                // Assigning values to variables

                commissionLimit =

                    commissionRecord.responseData.data.commissionLimit;

                commission = commissionRecord.responseData.data.commission;

            }

            // Perform Arithmetic Operations

            commissionLimit = parseInt(commissionLimit) + parseInt(amount);

            if (commissionLimit >= 20) {
                remainder = commissionLimit % 20;
                commission =
                    parseInt(commission) + Math.floor(commissionLimit / 20);
                commissionLimit = parseInt(remainder);
            }

            if (commissionRecord.responseData.exists === true) {
                //Update the document
                const result = await CommissionTable.updateCommissionNormal(
                    publicKey,
                    commission,
                    commissionLimit
                );

                if (result.success) {
                    return result;
                }
2

Can you be more specific? Do you have any examples? If a request comes for the same document you might lose some data since your code is not atomic. You read some data and then update it and then write it to the server. It is normal behaviour.

1 Like
3

Two processes read the same document and they perform some arithmetic operation and then update the document. So what happens is that one of the process will update first and then when second process updates the document, the changes made by the first process is lost.

5

Yes because your code does not prevent it. There is no way for the server to know what is your intent. Even me, I do not know.

What do you want the server to do? Once you know you have to modify your code so that it supports your requirements.

6

@steevej

How would one go about handling that kind of situation?

What pattern/concept would you consider?

My thoughts

To be honest unless you soft locked the record to provide a read block, and tested for that soft-lock on the record, this could happen on any database. Could end up being a bit of a blocker?

Or maybe if the time stamp on the record, if that has changed, then reload the record. Updating the application time stamp and applying the changed value to this new updated value. Hopefully finding a window where you can squeeze in your value. Again could be a bit of a process blocker, if that product was hot.

Or maybe, write all value updates to a separate table, not back to the record holding the data, and let another process update the value via this other table, but that one process does the writing, and how often it executes depends on the frequency of updates.

BTW I don’t have extensive experience of many databases, but I am not familiar with any which block reads.

7

Assume you have 2 threads (T1 and T2) reading the same document such as:
{ _id:1 , value: 2 }

Both do a calculation on value. T1 ends up with new_value:7 and T2 ends up with new_value:9.

The both issues an update with:

...updateOne(
  { _id:1 , value:2 } ,
  { $set : { value : new_value } } )

Only one of the two updates will succeed since the first update will have set value to its own value (7 or 9). It is impossible to tell which one will succeed. It is impossible for me to tell which one should succeed since it depends on the application. But you know which one failed by handling the returned value of updateOne. It then depends on the application if you re-read the document to create a new update.

In come cases, you want both so you do not put value:2 in the query part. For example, where you want to count page clicks. You would use $inc and then you do not care which one gets executed first because you had 2 clicks and you want both update to succeed.

1 Like
8

@steevej

I like that, make the previous value part of the update search, better than all the work mine would have involved.

And for the second one - idempotent :), don’t get to use that one in conversation very often.

9

The following is also a good approach.

This is also good depending on the application. The famous depending on the application.

Definitively: