We have a requirement where we want to use an “aggregation pipeline” which will include operations over 2 or 3 “source collections” and at the final stage of the pipeline the results has to be added to a “Target Collection”.
Now, the data in the source collections can change on a daily basis and therefore we would want the data inside “Target Collection” also to be Refreshed periodically. On checking the article On-Demand Materialized Views — MongoDB Manual it is mentioned that we can use a function to trigger the pipeline again, however there is no info on how this refresh can be scheduled. So we have the following questions :
- Could you kindly suggest how we can schedule the refresh of Target collection on a periodic basis using standard MongoDB instance, we dont have Atlas instance.
- If some JSON records insert/updates to the Target Collection are rejected during Aggregation Pipeline run then how can we get a summary of this information at end of run?
MongoDB version : 4.2.17
Hosted on: AWS EC2
Config: Standalone 3 node replica set
Thanks in advance.