Hi, is there any convenient way to monitor lag/delay between oplog and kafka source connector? We got an incident of losing offset due to oplog shrunk window, so now we would like to monitor it.
What tool would you like to use to monitor ? Right now everything is logged through the kafka connect logs.
We using prometheus and grafana, but I’m not sure that there’s any way to get the metrics I want (oplog cursor vs source connector current offset)
The mongodb connector uses Change Streams not the oplog, are you wanting to monitor the delta between oplog and change stream and/or changestream and its message in a kafka topic?
Changestream last item and currently processed item by source connector, to be able to see if there’s any lag