$lookup using $documents instead of 'from'

Working with Data
1

The docs says I can perform a $lookup without the ‘from’ field and passing an array to $documents stage inside a pipeline. But I’m getting an error trying that: “MongoServerError: missing ‘from’ option to $lookup stage specification”.

Is that really supported?

reference: https://www.mongodb.com/docs/manual/reference/operator/aggregation/documents/#use-a--documents-stage-in-a--lookup-stage

My use case is I have to perform an aggregation in one collection using $graphLookup to build an array with the document ancestors. That is in one collection called ‘inventoryCategories’
Now I need to get this data and do a $lookup in the “inventory’ collection to add that information in each document.
The solution I imagined was to perform the first aggregation on ‘inventoryCategories’” save the data on a array, and use this array in the second aggregation in the ‘inventory’ collection. That makes sense?

InventoryCategory:

[{
 _id: {
      "$oid": "641c81364e238225b6362cfb"
    },
parent:  {
      "$oid": "641cd6eb5d09817ea0222ecc"
    },
name: "Towable",
...other fields
}]

Array after first aggregation

[{
{
    "_id": {
      "$oid": "641c81364e238225b6362cfb"
    },
    "inventoryCategory": "Towable",
    "ancestors": [
      "Boom Lifts",
      "Material Handling"
    ]
  },
}]

Inventory:

[{
{
  "_id": {
    "$oid": "6425b1f0f7c12b4f67bdbd51"
  },
  "inventoryCategoryId": {
    "$oid": "641c81364e238225b6362cfb"
  },
...other fields
}]

Desired output (Materialized View - Inventory)

[{
{
  "_id": {
    "$oid": "6425b1f0f7c12b4f67bdbd51"
  },
  "inventoryCategoryId": {
    "$oid": "641c81364e238225b6362cfb"
  },
 "inventoryCategory": "Towable",
    "ancestors": [
      "Boom Lifts",
      "Material Handling"
    ]
}]

Aggregations:

 const pipeline = [
    {
      $graphLookup: {
        from: 'inventoryCategories',
        startWith: '$parent',
        connectFromField: 'parent',
        connectToField: '_id',
        depthField: 'order',
        as: 'ancestor',
      },
    },
    {
      $unwind: '$ancestor',
    },
    {
      $sort: {
        _id: 1,
        'ancestor.order': 1,
      },
    },
    {
      $group: {
        _id: '$_id',
        name: { $first: '$name' },
        parent: { $first: '$parent' },
        ancestor: { $push: '$ancestor' },
      },
    },
    {
      $project: {
        inventoryCategory: '$name',
        ancestors: '$ancestor.name',
      },
    },
  ];

  const categoriesWithAncestors = await categories.aggregate(pipeline).toArray();

  const inventoryPipeline = [
    {
      $lookup: {
        localField: 'inventoryCategoryId',
        foreignField: '_id',
        as: 'category',
        pipeline: [
          {
            $documents: [...categoriesWithAncestors],
          },
        ],
      },
    },
    {
      $set: {
        inventoryCategory: { $arrayElemAt: ['$category.name', 0] },
      },
    },
    {
      $unset: 'category',
    },
    {
      $limit: 5,
    },
   {
      $merge: {
        into: 'inventoryWithCategory',
        whenMatched: 'replace',
        whenNotMatched: 'insert',
      },
    },
  ];
2

First of all, you should never group on $_id - that’s an antipattern you can just sort the array in each document.

As far as $documents inside $lookup it absolutely works but $documents accepts any valid expression that resolves to an array of objects. and I can’t tell from your post, what is [...categoriesWithAncestors] - it seems like it’s a local variable?

What version are you using?

Asya

3

This is a bit strange - I cannot reproduce the error you say you are getting - are you sure this exact syntax gave you the missing ‘from’ option error? Are you using version 5.0 or older by chance? $documents was only introduced in 5.1/6.0 so maybe the parser is giving that error before getting to unrecognized stage inside $lookup?

4

If you are on older version and don’t have $documents you can emulate the same thing you are doing by defining a read only view on the pipeline that does the $graphLookup. Now you can just specify the view name in the from field of $lookup and it should all work.

Asya

5

Yes, is a local variable where I stored an array with the output of the previous aggregation.
You are totally right, I’m using atlas shared cluster and didn’t realize it only supports mongo 5.0

6

how can I use a read only view? It will not consume resources creating a view every time I run the aggregation?

7

The docs says that the results of $graphLookup are not in order, and I need the ancestors list to be ordered. How can I do that without grouping by _id?
Thanks for the help! I didn’t know it was an anti-pattern. Where can I found more information about patterns and anti-patterns?