$TopN returning partial results

Hi there,

I’m playing around with a $group stage that includes a $topN expression in MongoDB Compass → Aggregations tab, and C# code:

[
  {
    $sort: {
      JobName: 1,
    },
  },
  {
    $group: {
      _id: "$JobName",
      documents: {
        $topN: {
          output: "$$ROOT",
          sortBy: {
            CreatedTime: 1,
          },
          n: 10,
        },
      },
    },
  }
]

This works great and as expected. However, when I change “n” to some larger (obscene) number, I’m getting partial results and no error. Effectively, I’m telling it to give me all documents by making “n” so large, but I get about half of what I expect with no error. Additionally, in Compass, the explain times out whenever I ask for it with an error message of “$topN used too much memory and spilling to disk cannot reduce memory consumption any further. Memory limit: 104857600 bytes”. I’m unsure how to include “useDisk” in compass, however, I did include it in my C# code and I always get partial results – roughly half of what I expect.

Am I misunderstanding what “$topN” is capable of? I understand that such a large “n” number may seem excessive, but I’d expect some sort of error instead of partial results. As far as documentation goes, I’ve not found anything that provides limitations such as I’m running into.

Try the same aggregation in the shell - Compass will limit results in some cases, where the shell will return all the results. Let us know if you’re seeing the same issue in the shell. However, note that the document limit of 16MBs will prevent you from having top “billion” results (they are, after all, returned as an array inside the document).

Asya