If you’re using another cloud service like AWS you could look to utilise something like Amazon EventBridge (Serverless Event Router – Amazon EventBridge – Amazon Web Services & https://docs.atlas.mongodb.com/triggers/eventbridge/) or Step functions (Serverless Workflow Orchestration – AWS Step Functions – Amazon Web Services)
We have done something like this to kick start lots of machine learning containers.
We essentially do the following…
- MongoDB scheduled trigger starts
- MongoDB Realm function gets all data for the job it needs from MongoDB database
- Calls a Lambda function with the info of the job from realm function
- Lambda function calls X step functions
- Step functions call AWS Batch to do lots of grunt work and spin up containers
- At the end of the step function, another Lambda is called to write results to MongoDB
All of this uses just 1 MongoDB Realm function which starts the whole chain of events off.
Thanks