Java
MongoDB Developer Centerchevron-right
Developer Topicschevron-right
Languageschevron-right
Javachevron-right

Java - Aggregation Pipeline

Maxime BeugnetPublished Feb 01, 2022 • Updated Sep 23, 2022
MongoDBAggregation FrameworkJava
Copy Link
facebook icontwitter iconlinkedin icon
random alt
Rate this quickstart
star-empty
star-empty
star-empty
star-empty
star-empty

Updates

The MongoDB Java quickstart repository is available on Github.
March 25th, 2021
  • Update Java Driver to 4.2.2.
  • Added Client Side Field Level Encryption example.
October 21th, 2020
  • Update Java Driver to 4.1.1.
  • The Java Driver logging is now enabled via the popular SLF4J API so I added logback in the pom.xml and a configuration file logback.xml.

What's the Aggregation Pipeline?

Java badge
The aggregation pipeline is a framework for data aggregation modeled on the concept of data processing pipelines, just like the "pipe" in the Linux Shell. Documents enter a multi-stage pipeline that transforms the documents into aggregated results.
It's the most powerful way to work with your data in MongoDB. It will allow us to make advanced queries like grouping documents, manipulate arrays, reshape document models, etc.
Let's see how we can harvest this power using Java.

Getting Set Up

I will use the same repository as usual in this series. If you don't have a copy of it yet, you can clone it or just update it if you already have it:
If you didn't set up your free cluster on MongoDB Atlas, now is great time to do so. You have all the instructions in this blog post.

First Example with Zips

In the MongoDB Sample Dataset in MongoDB Atlas, let's explore a bit the zips collection in the sample_training database.
As you can see, we have one document for each zip code in the USA and for each, we have the associated population.
To calculate the population of New York, I would have to sum the population of each zip code to get the population of the entire city.
Let's try to find the 3 biggest cities in the state of Texas. Let's design this on paper first.
  • I don't need to work with the entire collection. I need to filter only the cities in Texas.
  • Once this is done, I can regroup all the zip code from a same city together to get the total population.
  • Then I can order my cities by descending order or population.
  • Finally I can keep the first 3 cities of my list.
The easiest way to build this pipeline in MongoDB is to use the aggregation pipeline builder that is available in MongoDB Compass or in MongoDB Atlas in the Collections tab.
Once this is done, you can export your pipeline to Java using the export button.
After a little code refactoring, here is what I have:
The MongoDB driver provides a lot of helpers to make the code easy to write and to read.
As you can see, I solved this problem with:
  • A $match stage to filter my documents and keep only the zip code in Texas,
  • A $group stage to regroup my zip codes in cities,
  • A $project stage to rename the field _id in city for a clean output (not mandatory but I'm classy),
  • A $sort stage to sort by population descending,
  • A $limit stage to keep only the 3 most populated cities.
Here is the output we get:
In MongoDB 4.2, there are 30 different aggregation pipeline stages that you can use to manipulate your documents. If you want to know more, I encourage you to follow this course on MongoDB University: M121: The MongoDB Aggregation Framework.

Second Example with Posts

This time, I'm using the collection posts in the same database.
This collection of 500 posts has been generated artificially but it contains arrays and I want to show you how we can manipulate arrays in a pipeline.
Let's try to find the three most popular tags and for each tag, I also want the list of post titles they are tagging.
Here is my solution in Java.
Here I'm using the very useful $unwind stage to break down my array of tags.
It allows me in the following $group stage to group my tags, count the posts and collect the titles in a new array titles.
Here is the final output I get.
As you can see, some titles are repeated. As I said earlier, the collection was generated so the post titles are not uniq. I could solve this "problem" by using the $addToSet operator instead of the $push one if this was really an issue.

Final Code

Wrapping Up

The aggregation pipeline is very powerful. We have just scratched the surface with these two examples but trust me if I tell you that it's your best ally if you can master it.
I encourage you to follow the M121 course on MongoDB University to become an aggregation pipeline jedi.
If you want to learn more and deepen your knowledge faster, I recommend you check out the M220J: MongoDB for Java Developers training available for free on MongoDB University.
In the next blog post, I will explain to you the Change Streams in Java.

Copy Link
facebook icontwitter iconlinkedin icon
Rate this quickstart
star-empty
star-empty
star-empty
star-empty
star-empty
Related
Tutorial
Integrating MongoDB with Amazon Managed Streaming for Apache Kafka (MSK)

May 26, 2022
Tutorial
How to migrate from Realm Java SDK to Realm Kotlin SDK

May 09, 2022
Tutorial
Kafka to MongoDB Atlas End to End Tutorial

May 13, 2022
Quickstart
Java - Change Streams

Sep 23, 2022
Table of Contents