Hello Team,
We had an issue a few days ago in production where data was stuck in Kafka, and those data were moved to Mongo DB in bulk (in general, the data was limited before to Mongo DB), which caused a change in plan, and it seems like that process took all the write tickets available in Mongo to process.
We try to produce the same scenario in our development setup, and after improving the query by removing the nulls and adding a proper index, we still see that the ticket count at the end reaches 0.
the job process is basically a k8s job deployment which inserts those kafka records into two collections and updates one field of the collection for new inserted documents
can you provide some guide whats wrong ?
Regards
Dash