Hi,
we are fairly experienced with MongoDB, but still wonder what experiences the community has made in using change streams.
Our scenario is that we are using change streams for event driven architectures, currently via Realm Triggers, but we are facing issues in Triggers being suspended when a bit of heavy writing occurs.
We have changestreams open for around 50 collections, no match expressions and in many cases full lookups, one collection with the preimage option set.
We are now considering moving away from Realm Triggers and consuming the change streams via serverfull instances dedicated for that purpose. We are using Node.js.
Questions:
- Shall we consume better the “whole changestream” instead of one changestream per collection and than let the consumer fan out into the different target destination. So 1 changestream across all collections vs 50 change streams.
- I guess one Node.js instance might be overwhelmed, what are good experiences for such a heavy consumer? Go? Rust? JVM-based ones?
Thanks for any input.