Data API now fails with A pipeline with this stage must be run as a System User

[Keeton_Hodgson]

I have a query to our DB that was working until yesterday, using the aggregate endpoint for Data API. https://data.mongodb-api.com/app/data-{app_id}/endpoint/data/beta/action/aggregate

It has a $merge stage in this aggregation pipeline.

This query was working just fine in both our prod and staging environments for several weeks until yesterday. It now fails with the following error:

“Failed to aggregate documents: FunctionError: aggregation stage "$merge" is not supported in a user context. A pipeline with this stage must be run as a System User”

We have changed nothing on our end. Could you please advise on how to either make our Data API token run as a system user or how to run an aggregation pipeline now using the Data API?

Thank you.

[Sumedha_Mehta1 (Sumedha Mehta)]

Hi Keeton - the preview version of the data API was modified slightly to improve our security by forcing application auth due to its ability to open up data in the cluster, however you can still create an endpoint to run the merge pipeline.

You can do the folllowing:

1. Navigate to the Data API ‘Advanced Settings’
2. Create a Custom HTTPS endpoint
3. Use the following function:

exports = async function (request, response) {
  
   const mongo = context.services.get(<your cluster name>);
   collection = mongo.db(<db name>).collection(<collection>);
   documents = await collection.aggregate(pipeline).toArray();
   return {documents}

Since $merge is only available as a system user, you will have to set the associated endpoint as a ‘System’ endpoint, and then do the authentication either within the function (i.e.

request.header["api-key"] = <your SECRET API KEY>

)

Or you can create another wrapper system function that simply invokes the above function that runs as ‘application auth’ it will respect the need for proper API key authentication, but the invoked function will run as system.