Hi guys, we want to know if you can help us with a question.
Is there a way to send an “aggregate pipeline” instruction from my application’s backend to be processed asynchronously by Mongo and then go and check its status and determine if it is still running or is finished. All of the above focused on the use of the Mongo driver, more specifically the .net Core driver.
Currently we are executing an aggregation pipeline that takes more than 5 minutes and we are invoking it from an azure function that has an execution limit of 5 minutes, so the function dies before the mongo processes the instruction.
Depending on the use case, 5 minutes seems like an excessive time. Do you know why the aggregation pipeline process is taking this long to be processed ?
If the complication is mainly due to the document schema, please consider to re-design the schema to fit the use case. See also Building With Patterns: A Summary
You could also try to split the aggregation pipeline into smaller batches. For example, by using $merge operator and materialised views. Alternatively, you could also split the aggregation pipeline into smaller tasks.