It’s managed service ( AWS emr)
In dev environment (the one I am using right now is a community one) but we do have Atlas as well.
Yes the data is getting updated frequently.
Data is getting updated via our web application.
So far we were using debizium connectors to push the data to Kafka but now since mongo db provides steaming, we wanted to try it out.
Problem is as soon as I am applying any action in foreachbatch sink ( further transformation is written in in foreach batch to load the data into data warehouse) it gives me NPE.