Will change streams with fullDocument=updateLookup lose new image data when update very frequently?

Change stream uses DocumentSourceLookupChangePostImage pipeline stage to send a new query to MongoD from MongoS when set {fullDocument:updateLookup}. But check out this scenario: users write so many update sentences with $set and $unset, so each one needs to send a new query to MongoD to get the new image with given _id and shard_key:

ts=1, update: {a:1}, {$set:{b:1}}
ts=2, update: {a:1}, {$set:{b:2}}
ts=3, update: {a:1}, {$set:{b:3}}
ts=4, update: {a:1}, {$set:{b:4}}

When MongoS receives the change stream events of ts=1, update: {a:1}, {$set:{b:1}} from MongoD, and then send query to get the new image of {a:1}, however, when MongoD receive this query command, the ts=2, update: {a:1}, {$set:{b:2}} has already been applied, so what’s the result of new image? {a:1, b:1} or {a:1, b:2}? In my test, the result is {a:1, b:1}, but I wonder how does change stream ensure consistency?

I also have another question about change stream, hope to get help here:

Before MongoDB 4.2, the read concern level must be majority. But starting in MongoDB 4.2, change streams are available regardless of the majority read concern support. So my question is when the read concern level is available or local, how to ensure the correctness of the data when rollback happened? Or it just depends on the users setting, users should keep in mind different read concern level will lead to different result?

Hi @Vinllen_Chen,

The change stream events are based on the order of documents in the oplog. If your updates resulted in 2 different oplog entries they should be returned as 2 events ordered by the time they are placed in the oplog.

Regarding your second question I believe you are correct, in 4.2 its possible that events read with lower readconcern will be rolled back.

Let me know if you have any further questions.


About the first question, after i went through the source code, I wonder how does the change streams ensure consistency if I update one document several times? Because the query in DocumentSourceLookupChangePostImage stage in MongoS is different of DocumentSourceOplogMatch in MongoD, and there exists time gap.