Oplog tailing on MongoDB atlas sharded cluster

[Manish_Rawat1 (Manish Rawat)]

In a sharded setup on MongoDB Atlas, We do not have network access to shard replicaset. This blocks the oplog tailing feature of MongoDB.

Is there a way to bypass this limitation

[chris (Chris Dellaway)]

For most use case this has been replaced by Change Streams

[Manish_Rawat1 (Manish Rawat)]

Hi Chris,
Thanks for your response.

However, in a big cluster with 100s of shards. Processing all the changes from a single stream will not be performant for us. Also, our application is based on an Oplog tailing-based Kafka connector. Converting to changestream is not a trivial change for us.

[kevinadi (Kevin Adistambha)]

Hi @Manish_Rawat1

Note that you can open a changestream for a collection, a database, or the whole deployment so you can fine-tune your needs. Additionally, the official Kafka connector can be configured to use changestream.

Having said that, it sounds like you have a very large deployment in Atlas. In this case, you might want to contact Atlas support for suggestions, since they will have better visibility and insight into your exact situation.

Best regards
Kevin