Change Stream with changable pipeline

Hi Guys,

I have an application that uses a change stream to watch incoming information. However, there is a use case that pipeline of the change stream will be updated from other sources. I cannot predict the input, it is user related.

Is there a suggested way to update the pipeline for a change stream? I am using mongocxx. Mongocxx example is mostly welcomed, other language/general suggestions are also appreciated.

thanks.

it might be a rare case as i cannot find much information about this problem. however, i think it could a case people will run into.

ok, there is no answer which means it is either not a problem, or it is an unsolvable problem…i will do some work around then lol

Hi @Kong_Ip welcome to the community!

Currently I don’t believe there is a way to change the pipeline other than creating a new changestream. From the Python example in https://www.mongodb.com/docs/manual/changeStreams/

pipeline = [
    {"$match": {"fullDocument.username": "alice"}},
    {"$addFields": {"newField": "this is an added field!"}},
]
cursor = db.inventory.watch(pipeline=pipeline)
document = next(cursor)

The pipeline is part of the creation of the change stream itself, so I don’t think it can be modified once it’s created.

An alternative would be to get everything from the change stream (without any pipeline), pass the output to a second process that can filter it, then pass that output to the final destination. You can design the second process to have modifiable filters (which could be in any language).

Best regards
Kevin

3 Likes

thanks very much Kevin.

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.