QAing New Code with MMS: Map/Reduce vs. Aggregation Framework

This is part three of a three-part guest series by Alex Giamas, Co-Founder and CTO of CareAcross.

• Part 1: The Top 5 Metrics to Watch in MongoDB
• Part 2: Setting Up Actionable Alerts and Procedures in MMS

When releasing software, most teams focus on correctness, and rightly so. But great teams also QA their code for performance. MMS Monitoring can also be used to quantify the effect of code changes on your MongoDB database. Our staging environment is an exact mirror of our production environment, so we can test code in staging to reveal performance issues that are not evident in development. We take code changes to staging, where we pull data from MMS to determine if feature X will impact performance.

As a working example, we can use MMS to calculate views across a day using both Map/Reduce and the aggregation framework to compare on their performance and how they affect overall DB performance.

Our test data consists of 10M entries in a collection named views in the database named CareAcross with entries of the following style:

{
userId: “userIdName”, date: ISODate(“2013-08-28T00:00:01Z”), url: “urlEntry”,  
}

Using a simple map reduce operation we can sum on our documents values and calculate the sum per userId:

 db.views.mapReduce(function () {emit(this.userId, 1)}, function (k,v) {return Array.sum(v)}, {out:"result"})

The equivalent operation using Aggregation framework looks like this:

db.views.aggregate({$group: {_id:"$userId", total:{$sum:1}}})

The mapReduce function hits the server at 18:54. The aggregation command hits the server at 19:01.

If we compare these two operations across our data set we will get the following metrics from MMS:

In terms of lock percentage, we see that the map reduce operation started locking up our test DB, whereas the aggregation framework’s impact is insignificant, primarily because of the fact that aggregation happens in memory.

This can be seen in the next graph, memory consumption:

As is fairly obvious, the first spike from 70MB to 0.54GB of usage occurred at the invocation of mapreduce at 18:54 and then another spike to 1.07GB of resident memory usage happened when the aggregation framework passed through the pipeline all of our views records, grouped by userId at 19:01.

In terms of performance, the map/reduce operation took 139 seconds, whereas the equivalent aggregation framework operation was completed in 52 seconds, which is really an improvement.

As for cursors, there were 2 open cursors for the mapreduce operation and 1 for the aggregation framework operation and that’s a something that we would like to monitor in case we are stuck with several open cursors dragging our database.

From the graphs at the online console, you can see that aggregation is trading off memory for speed, lock percentage time and resource utilization in general. MMS can help us visualize the difference and identify any abnormal behaviors.

Takeaways from the series

• Set up monitoring and alerting, but also have procedures for for when those alerts arrive.
• Carefully calibrate alerts - each email or text message should be actionable.
• Install MMS on your staging or development environment and use it to test performance of new features.

If all these sound interesting to you and want to deal with large scale MongoDB powered applications in RoR, we are hiring! We are a stealth mode, digital health oriented company based in the UK with operations in Athens, Greece.

Thanks to Alex for sharing his expertise on the MongoDB blog in this series!