Run materialized views refresh from OS scheduler

[Tamar_Nirenberg]

Hi,

I use Mongo 4.2 on-premise.

I plan to use the $merge in order to create materialized views for some heavy aggregations.

I saw that Mongo Atlas has built-in triggers to refresh the materialized views automatically.

Since I cannot use this - can I simply use a script that will run the aggregation with the $merge from an OS scheduler (such as cronjob) every, say, two hours?

Am I missing something?

Thanks,
Tamar

[Abdullah_Madani]

@Tamar_Nirenberg

Since I cannot use this - can I simply use a script that will run the aggregation with the $merge from an OS scheduler (such as cronjob) every, say, two hours?

Yes definitely you can! First, you need to create a script file that can be called by mongo shell, without entering the console. The syntax to call the script depends on which Mongo shell you are using. If you are using mongosh (recommended) then follow the below steps:

1. Copy the aggregate command in a script file, say aggregate_pipeline.js file, and save it

2. Test the script by executing it from the Linux shell as follows:
   mongosh --host 127.0.0.1 --port 27017 --username myuser --password superSecret --file aggregate_pipeline.js

3. If the above step succeeds, create a bash shell file in the same directory and type in the above command, and save it as .sh
   Don’t forget to grant execute privilege to the bash script (chmod u+x command.sh)

   Example contents of the bash file:

   #!/bin/bash
   mongosh --host 127.0.0.1 --port 27017 --username myuser --password superSecret --file aggregate_pipeline.js

4. Configure the cronjob for the above bash script to run every two hours, as below:

→ crontab -e
→ 0 */2 * * * sh <path/to/bashScript.sh>
→ save and exit

If you are using old mongo shell (deprecated), then use the below command in the bash script:

mongo -u username -p password --authenticationDatabase auth_db_name --eval ‘<aggregate_pipeline_command>’ myDbName

Note: if you are using Mongo operators starting with a $ sign, you’ll need to surround the eval argument in single quotes to keep the shell from evaluating the operator as an environment variable.

Best Regards,
Abdullah Madani.

[Tamar_Nirenberg]

Thank you @Abdullah_Madani for clarifying.

I am actually very familiar mongo shell and cronjobs, I was just wondering if the Mongo Atlas solution for Materialized views refresh has some hidden advantage over a simple cronjob.
If it does - I was wondering how to do the same with a Mongo on-premise.
If all not possible - then I will have to use a cronjob I guess.

Thanks,
Tamar