MongDB Compass: operation exceeded time limit

Working with Data
,
1

I have two Collections:
asset_earnings collection with 524K documents
track_id collection with 944 documents
this aggregation is run on track_id
I have the Max Time MS set to 2147483647 and the following aggregation still times out:
Even the lookup alone also timed out.

[
  {
    $lookup:
      /**
       * from: The target collection.
       * localField: The local join field.
       * foreignField: The target join field.
       * as: The name for the results.
       * pipeline: Optional pipeline to run on the foreign collection.
       * let: Optional variables to use in the pipeline field stages.
       */
      {
        from: "asset_earnings",
        localField: "_id",
        foreignField: "track_id",
        as: "output",
      },
  },
  {
    $unwind:
      /**
       * path: Path to the array field.
       * includeArrayIndex: Optional name for index.
       * preserveNullAndEmptyArrays: Optional
       *   toggle to unwind null and empty values.
       */
      {
        path: "$output",
        preserveNullAndEmptyArrays: true,
      },
  },
  {
    $project: {
      _id: 0,
      track_id: "$_id",
      year: "$output.reporting_year",
      payout: "$output.payable_amount",
    },
  },
  {
    $out:
      /**
       * Provide the name of the output collection.
       */
      "matches_with_details",
  },
]
2

if you do not have one yet, create an index on track_id and try again. without an index, every lookup operation needs a collection scan which takes longer.

also, try using another project stage before the unwind. the unwind makes copies of that full document as many times the size of the array. reducing the number of other fields will reduce the size of the document in memory and increase performance if there is a disk swap in use

1 Like