MongoServerError: WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction

Working with Data
, ,
1

I am getting the the following error when I try to update only 2 documents with the transaction:

MongoServerError: WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction.

From all the docs I read, this can be an issue when modifying more than 1000 documents, but since I am updating only 2, it should not throw any error.

That being said, this error only happens sometimes, not always. Usually, the logic will pass without the error, but sometimes it is thrown randomly.

MongoDB version: 5.0.14

Here is the code:

exports.setArticleView = async (req, res) => {
  const userId = req.user._id;
  const { article_id } = req.body;

  // Start a session
  const session = await mongoose.startSession();

  try {
    // Start a transaction
    session.startTransaction();

    await ArticleView.create([{ articleId: article_id, user: userId }], { session });

    await Article.updateOne({ _id: article_id }, { $inc: { views: 1 } }, { session });

    // Commit the transaction
    await session.commitTransaction();

    return res.status(200).json({ success: true });
  } catch (error) {
    return res.status(500).json({ success: false });
  } finally {
    // End session
    session.endSession();
  }
};

I think it’s the following scenario when the error happens:

  1. One user view the article and transaction starts for that user.
  2. Second user view the article after a few milliseconds (almost the same time) after the first user.
  3. Since transaction of the first user didn’t finish, the transaction for the second user aborts.

I assume this based on the following paragraph from the docs:

If a multi-document transaction is in progress, new DDL operations that affect the same database(s) or collection(s) wait behind the transaction. While these pending DDL operations exist, new transactions that access the same database(s) or collection(s) as the pending DDL operations cannot obtain the required locks and and will abort after waiting maxTransactionLockRequestTimeoutMillis (which defaults to 5ms).

I think it can probably be solved by increasing the maxTransactionLockRequestTimeoutMillis , but I wanted to check with the community. If I would increase maxTransactionLockRequestTimeoutMillis config, would that have some further implications?

1 Like
2

Just a guess, but I’d think you’d have to commit the created document before you try to update it.

3

Hi @Jack_Woehr, thanks for the answer!

I am not sure what do you mean? I am creating document in one collection, and then updating document from other collection. It’s not like I am updating the same document that I created (even though I think that should work too).

That being said, this error only happens sometimes, not always. Usually, the logic will pass without the error, but sometimes it is thrown randomly. (I just figured I didn’t mention this in the question itself and I think it’s important. Will updated the question).

4

is the system actually in production? Is other activity taking place?

5

@Jack_Woehr Yup, it’s in production. No, this is the whole business logic in that endpoint.

6

Well, there certainly are a lot of caveats about transactions!
Unless someone posts with an answer, I suggest you read carefully both of the following:

If you don’t find anything there, you may need to file an issue.

1 Like
7

Thanks for the references! I already read them, but I couldn’t figure out what could be the exact issue.

I assume it’s maybe the following scenario:

  1. One user view the article and transaction starts for that user.
  2. Second user view the article after a few milliseconds (almost the same time) after the first user.
  3. Since transaction of the first user didn’t finish, the transaction for the second user aborts.

I assume this based on the following paragraph from the docs:

If a multi-document transaction is in progress, new DDL operations that affect the same database(s) or collection(s) wait behind the transaction. While these pending DDL operations exist, new transactions that access the same database(s) or collection(s) as the pending DDL operations cannot obtain the required locks and and will abort after waiting maxTransactionLockRequestTimeoutMillis (which defaults to 5ms).

I think it can probably be solved by increasing the maxTransactionLockRequestTimeoutMillis, but I wanted to check with the community too.

8

@steevej do you know the answer to @NeNaD 's question?

9

Thanks for tagging me. I have seen the post but have nothing to add at this moment. I still have not enough experience with transactions to have experimented this issue. In addition, it is with mongoose which I avoid.

The only thing I would do it to try without mongoose. The issue might be mongoose.

And as I write, I think that this code can should be done outside a transaction. You just create the ArticleView and if successful you $inc the Article/views. $inc is already atomic so if 2 ArticleView are created at the same time, which is also atomic, the count in $inc twice. The issue with transaction in this case, I think, is that you have 2 transactions starting at the more or less the same time. Both transaction take a snapshot of Article. One of the transaction terminate correctly. But now the snapshot of Article of the 2nd transaction is stale as it has been updated by the first transaction.

I thought I had nothing to write. Thanks for letting me know I had.

1 Like
10

Hi @steevej,

Thanks for the response, and @Jack_Woehr thanks for tagging him.

I don’t think the issue is related to Mongoose at all, since Error clearly say it’s MongoServerError. Also, I don’t want do it without a transaction because I am tracking unique user views, so it could happen that ArticleView will be created and later it would fail to increase the article’s view count, which would lead to data inconsistency. Thank you for the answer though. :smiley:

11

@Stennie_X, @MaBeuLux88, @Pavel_Duchovny, @Jason_Tran

Can you please let me know your opinions on this if you don’t mind.

Thank you in advance! :smiley:

12

The only way

is that if the code crashes between the ArticleView creation and the Article.views increment.

But if you want to stick with transaction you have to do what the error message tells you to do

13

Yup, I will implement retry logic, it’s just this error should not be thrown in the first place in my opinion, and that is why I raised the question.

I am trying to figure out if the maxTransactionLockRequestTimeoutMillis config is the reason for the error, and if the error can be prevented by increase it (if yes, would that have some further implications…). :smiley:

14

I beg to differ. Under load you are most likely to have 2 transactions started at almost the same time. Especially if you run multiple instances of your code. This is what the error is about. You have 2 transactions wanting to update the same resources, one have to wait so that the data is consistent at end.

You decrease the timeout you increase the chance of having the error but you must retry more operations increasing the load on the server and reducing the capacity of the client.

You increase the timeout you decrease the chance of having the error but you have more transactions awaiting which increases the load on the server and reduce the capacity of the client.

My conclusion is the same. Doing it without transactions will provide the same consistency and better throughput.

15

i’m also getting the same error and i have same scanario as you have. but after reading all the replys i don’t think so that there is a better solution for that. If you found the solution then please let me know.
thanks😊

16

Hey :wave: @Muzammal_Hussain,

Thanks for reaching out to the MongoDB Community.

I recommend creating a new post with your specific error message, the accompanying code snippet, the query you are attempting, and details about the MongoDB version you are using. Additionally, please provide the driver version you are using.

These details will help the community members better understand and troubleshoot the issue you’re facing. Feel free to share any other relevant information that might be useful.

Meanwhile, please refer to the documentation to learn more about the MongoDB Transactions:

Best regards,
Kushagra