Why would first aggregate query be so slow and rest fast everytime?

Been recently wrangling some huge collections with MongoDB (connection to each of the RS) and was wondering why every time I open a new connection to make an aggregation (always the same aggregation) the first one requires a minute or so to retrieve data, and subsequent aggregations take 5 seconds or so to complete.

Then, if I close the connection with MongoDB secondary and straight away reconnect to the same or connect to another secondary, same thing happens. Really slow first aggregation but quicker afterwards.

Needless to say that the aggregation is using an index.

Why is this happening? Didn’t Mongo cache the result set in memory for subsequent calls? Not like the aggregation result but the portion of data used.

I am not too sure but I suspect that the working set of one secondary does not necessaraly match the working set of another secondary especially if you manually connect and read from secondaries.