Explore Developer Center's New Chatbot! MongoDB AI Chatbot can be accessed at the top of your navigation to answer all your MongoDB questions.

Join us at AWS re:Invent 2024! Learn how to use MongoDB for AI use cases.
MongoDB Developer
MongoDB
plus
Sign in to follow topics
MongoDB Developer Centerchevron-right
Developer Topicschevron-right
Productschevron-right
MongoDBchevron-right

Spring Data Unlocked: Advanced Queries With MongoDB

Ricardo Mello7 min read • Published Nov 08, 2024 • Updated Nov 08, 2024
SpringMongoDBAggregation FrameworkJava
FULL APPLICATION
Facebook Icontwitter iconlinkedin icon
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty
I bet you’re here because you’ve already read the first article in this series, Getting Started With Java and MongoDB. But even if you haven’t, don’t worry—you’re in the right place. In the first part of this series, we explored key concepts related to Spring and learned how to create a project and configure it to seamlessly integrate with MongoDB. This is the second part of the series Spring Data Unlocked, where we will continue working on our project and explore Spring Data's capabilities by creating advanced queries and seeing how seamlessly it can be done.

Pre-requisites

Refreshing our memory

If you don’t remember what we discussed earlier, don’t worry. Read the previous article on how to get started with Java and MongoDB or refresh your memory with our data model that we are working on:
1{
2 "id": "672182814338f60133ee26e1",
3 "transactionType": "Debit",
4 "amount": 888.0,
5 "currency": "USD",
6 "status": "In Progress",
7 "description": "Transfer to Ricardo",
8 "createdAt": "2024-10-09T14:00:00",
9 "accountDetails": {
10 "originator": {
11 "accountNumber": "2376543213",
12 "name": "Maria",
13 "bank": "Bank G"
14 },
15 "beneficiary": {
16 "accountNumber": "2234987651",
17 "name": "Ricardo Mello",
18 "bank": "Bank V"
19 }
20 }
21}
Our transaction model contains this structure, and we will proceed with it.

The MongoRepository

Still focusing on productivity and speed, we will continue using our MongoRepository interface and create a query to retrieve our transactions by type. Open the TransactionRepository class and add the following method:
1List<Transaction> findByTransactionType(String type);
As you can see, we are finding all transactions by the type. Spring, through naming conventions (derived queries), allows you to create queries based on the method names.
Tip: We can monitor the log process by enabling a DEBUG setting in the application.properties file:
1logging.level.org.springframework.data.mongodb=DEBUG
We will be able to see the query in the console:
12024-10-15T18:30:33.855-03:00 DEBUG 28992
2[SpringShop] [nio-8080-exec-6] o.s.data.mongodb.core.MongoTemplate:
3find using query: { "transactionType" : "Transfer"} fields: Document{{}} sort: { "transactionType" : "Transfer"} for class: class com.mongodb.Transaction in collection: transactions
Now, let’s say we want all transactions where the amount is greater than 3000:
1List<Transaction> findByAmountGreaterThan(double amount);
It is possible to delete records in the same way. Take a look:
1void deleteByTransactionType(String type);
If everything is working correctly, your code should be similar to this:
1package com.mongodb;
2import org.springframework.data.mongodb.repository.MongoRepository;
3import org.springframework.stereotype.Repository;
4import java.util.List;
5
6@Repository
7public interface TransactionRepository extends MongoRepository<Transaction, String>{
8 List<Transaction> findByTransactionType(String type);
9 List<Transaction> findByAmountGreaterThan(double amount);
10 void deleteByTransactionType(String type);
11}

@Query

Continuing our development, we can use the Query class that Spring provides instead of working with derived queries. This alternative is also interesting. Below is an example of how to fetch transactions by status, displaying (Projection) only the fields createdAt, accountDetail, and amount, and sorting by createdAt in descending order:
1@Query(
2 value= "{ 'status' : ?0 }",
3 fields="{ 'createdAt': 1, 'accountDetails' : 1, 'amount' : 1}",
4 sort = "{ createdAt: -1}"
5)
6List<Transaction> findByStatus(String status);

@Update

We can also perform update operations on our database using this annotation. In the first part, we apply the filter, and then we proceed to update the status.
1@Query("{ '_id' : ?0 }")
2@Update("{ '$set' : { 'status' : ?1 } }")
3void updateStatus(String id, String status);
Note that in the code above we are combining @Query and @Update.

@Aggregation

Another important annotation in Spring Data is @Aggregation. As the name suggests, it allows us to create aggregation stages. Let’s imagine the following scenario:
We want to return the total amount grouped by transaction type. To do this, we'll create the following method:
1@Aggregation(pipeline = {
2 "{ '$match': { 'transactionType': ?0 } }",
3 "{ '$group': { '_id': '$transactionType', 'amount': { '$sum': '$amount' } } }",
4 "{ '$project': { 'amount': 1 } }"
5})
6List<Transaction> getTotalAmountByTransactionType(String transactionType);
  • In the first stage, $match, we filter the transactions by the specified type.
  • In the second stage, $group, we group the transactions by transactionType and sum the values from the amount field.
  • Finally, in the $project stage, we display the total amount.
A new request from our product team has come in, and they want us to save all transactions with an error status in a new collection. We could set up a job to run once a day and trigger a method to handle this for us. For that, we can use MongoDB’s $out stage:
1@Aggregation(pipeline = {
2 "{ '$match': { 'status': 'error' } }",
3 "{ '$project': { '_id': 1, 'amount': 1, 'status': 1, 'description': 1, 'createdAt': 1} }",
4 "{ '$out': 'error_transactions' }"
5})
6void exportErrorTransactions();
After this method is executed, the aggregation will process, and any documents with an error status will be inserted into a new collection within the same database called error_transactions.
Let's break it down again:
  • $match to filter transactions with an error status
  • $project to specify which fields we want to include in the error collection
  • $out to define the collection where we will export the error transactions
Note: While we won’t go into the specifics of $out, I highly recommend reviewing its behavior before using it in a production environment.
As you can see, this annotation is very powerful and can provide features for increasingly complex queries. Let’s assume we now have a new collection called customers with some additional fields:
1{
2 "name": "Ricardo Mello",
3 "email": "ricardo.mello@mongodb.com",
4 "accountNumber": "2234987651",
5 "phone": "5517992384610",
6 "address": {
7 "street": "Street 001",
8 "city": "New York"
9 }
10}
We can use $lookup to perform a join between the transaction and customer collections using the accountNumber field and bring in extra values like phone and address.
1@Aggregation(pipeline = {
2 "{ '$lookup': { " +
3 "'from': 'customer', " +
4 "'localField': 'accountDetails.originator.accountNumber', " +
5 "'foreignField': 'accountNumber', " +
6 "'as': 'originatorCustomerDetails' } }",
7 "{ '$project': { " +
8 "'amount': 1, " +
9 "'status': 1, " +
10 "'accountDetails': 1, " +
11 "'originatorCustomerDetails': 1 } }"
12})
13List<CustomerOriginatorDetail> findTransactionsWithCustomerDetails();
In the code above, we’re using the $lookup operator to join the new customer collection through the accountNumber field, and returning a new list of CustomerOriginatorDetail.
1public record CustomerOriginatorDetail(
2 double amount,
3 String status,
4 Transaction.AccountDetails accountDetails,
5 List<OriginatorCustomerDetails> originatorCustomerDetails
6) {
7 private record OriginatorCustomerDetails(
8 String name,
9 String accountNumber,
10 String phone,
11 Address address) {
12
13 private record Address(String city, String street) {}
14 }
15}
Note: Remember, we're not focusing on data modeling here, just showing that it’s possible to work with $lookup in this context.

Custom aggregation

Another approach with the @Aggregation annotation is to create an interface that represents an aggregation, which simplifies its usage. Our SearchAggregate annotation enables the creation of reusable search functionality using MongoDB's aggregation framework, facilitating efficient and structured querying of text data in your database.
1import org.springframework.data.mongodb.repository.Aggregation;
2
3import java.lang.annotation.ElementType;
4import java.lang.annotation.Retention;
5import java.lang.annotation.RetentionPolicy;
6import java.lang.annotation.Target;
7
8@Retention(RetentionPolicy.RUNTIME)
9@Target({ ElementType.METHOD })
10@Aggregation(pipeline = {
11 "{ '$search': { 'text': { 'query': ?0, 'path': ?1 } } }"
12})
13@interface SearchAggregate {
14}
Now, to utilize our annotation in a method, we can pass the arguments for the query and path in the TransactionRepository class:
1@SearchAggregate
2List<Transaction> search(String query, String path);

PagingAndSortingRepository

An important aspect to note is that if we look through the interfaces extended by MongoRepository, we’ll find PagingAndSortingRepository, which includes the method:
Page<T> findAll(Pageable pageable);
This method is crucial for working with pagination. Let’s go back to our TransactionService and implement the following code:
1public Page<Transaction> findPageableTransactions(
2 Pageable pageable
3) {
4 return transactionRepository.findAll(pageable);
5}
Now, in the TransactionController, we can make the paginated call as follows:
1@GetMapping("/pageable")
2public PagedModel<Transaction> findAll(@RequestParam(defaultValue = "0") int page,
3 @RequestParam(defaultValue = "100") int sizePerPage,
4 @RequestParam(defaultValue = "ID") String sortField,
5 @RequestParam(defaultValue = "DESC") Sort.Direction sortDirection) {
6 Pageable pageable = PageRequest.of(page, sizePerPage, Sort.by(sortDirection, sortField));
7 return new PagedModel<>(transactionService.findPageableTransactions(pageable));
8}
And finally, we can call our method using the following curl command.
1curl --location 'http://localhost:8080/transactions?page=0&sizePerPage=10&sortField=description&sortDirection=ASC'

The MongoTemplate

A very interesting alternative to achieve flexibility and control over MongoDB operation is MongoTemplate. This template supports operations like update, insert, and select and also provides a rich interaction with MongoDB, allowing us to map our domain objects directly to the document model in the database. Let's start by creating a new MongoConfig class:
1package com.mongodb;
2
3import com.mongodb.client.MongoClient;
4import com.mongodb.client.MongoClients;
5import org.springframework.context.annotation.Bean;
6import org.springframework.context.annotation.Configuration;
7import org.springframework.data.mongodb.core.MongoOperations;
8import org.springframework.data.mongodb.core.MongoTemplate;
9
10@Configuration
11public class MongoConfig {
12
13 @Bean
14 public MongoClient mongoClient() {
15 MongoClientSettings settings = MongoClientSettings.builder()
16 .applyConnectionString(new ConnectionString("<your connection string>"))
17 .build();
18 return MongoClients.create(settings);
19 }
20
21 @Bean
22 MongoOperations mongoTemplate(MongoClient mongoClient) {
23 return new MongoTemplate(mongoClient, "springshop");
24 }
25}
We can observe the @Bean annotation from Spring, which allows us to work with the injected MongoTemplate in our services. Continuing with our development, we will work with the Customer class to manipulate it. To do so, create the Customer record:
1package com.mongodb;
2
3public record Customer(
4 String name,
5 String email,
6 String accountNumber,
7 String phone,
8 Address address
9) {
10 public record Address(
11 String street,
12 String city
13 ) {}
14}

Insert

Let's start with the basic insertion model and then evolve to other operations. To do this, we will create our CustomerService with the following method:
1import org.springframework.data.mongodb.core.MongoOperations;
2import org.springframework.stereotype.Service;
3
4@Service
5public class CustomerService {
6
7 private final MongoOperations mongoOperations;
8
9 CustomerService(MongoOperations mongoOperations) {
10 this.mongoOperations = mongoOperations;
11 }
12
13 public Customer newCustomer() {
14 var customer = new Customer(
15 "Ricardo",
16 "ricardohsmello@gmail.com",
17 "123",
18 "1234",
19 new Customer.Address("a", "Sp")
20 );
21
22 return mongoOperations.insert(customer);
23 }
24
25}
Our service will work with mongoOperations, which will handle the insertion of our document.
Note that I am creating (new Customer) to illustrate the insertion. A good practice would be to receive this Customer as an argument in our function.

BulkWrite

One way to handle bulk insertions is by using bulkWrite. The MongoTemplate provides us with the bulkOps method, where we can provide a list, and it will be executed in batches:
1public int bulkCustomerSample(List<Customer> customerList) {
2 BulkWriteResult result mongoOperations.bulkOps(BulkOperations.BulkMode.ORDERED, Customer.class)
3 .insert(customerList)
4 .execute();
5
6 return result.getInsertedCount();
7}

Query

Moving on to queries, we have the implementation of queries in the MongoTemplate class. In the code below, we are searching for a customer by email and returning only one:
1public Customer findCustomerByEmail(String email) {
2 return mongoOperations.query(Customer.class)
3 .matching(query(where("email").is(email)))
4 .one()
5 .orElseThrow(() -> new RuntimeException("Customer not found with email: " + email));
6}
The query method (imported statically) expects a Criteria (where), also imported statically, where we can apply various filters such as gt(), lt(), and(), and or(). For reference, see the Spring documentation.

Aggregation

Let’s imagine we need a query that returns the total number of customers by city. To do this, we need to group by city and count the number of customers. To achieve this goal, first, let's create a new record to handle it:
1public record CustomersByCity(
2 String id,
3 int total
4){}
And then, we will create a totalCustomerByCity method:
1public List<CustomersByCity> totalCustomerByCity() {
2
3 TypedAggregation<Customer> aggregation = newAggregation(Customer.class,
4 group("address.city")
5 .count().as("total"),
6 Aggregation.sort(Sort.Direction.ASC, "_id"),
7 project(Fields.fields("total", "_id")));
8
9 AggregationResults<CustomersByCity> result = mongoOperations.aggregate(aggregation, CustomersByCity.class);
10 return result.getMappedResults();
11}
The static method newAggregation offers us a good approach to manipulate our aggregation stages. We are grouping by city and using the count() function, which internally performs a sum on the total number of customers, sorting by _id in ascending order and displaying both the total and id fields.

Conclusion

In this second part of our series, Spring Data Unlocked, we explored how to create complex queries with MongoRepository and MongoTemplate. This tutorial covered concepts such as pagination, custom annotations, bulk inserts, and powerful aggregation queries.
The complete code is available in mongo-developer.
If you have any questions, feel free to leave them in the comments.
Top Comments in Forums
There are no comments on this article yet.
Start the Conversation

Facebook Icontwitter iconlinkedin icon
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty
Related
Quickstart

Basic MongoDB Operations in Python


Sep 23, 2022 | 11 min read
Article

Massive Arrays


Oct 01, 2024 | 4 min read
Tutorial

Confessions of a PyMongoArrowholic: Using Atlas Vector Search and PyMongoArrow to Semantically Search Through Luxury Fashion Items


Aug 09, 2024 | 9 min read
Tutorial

How to Migrate Your Flask App From SQL To MongoDB


Jun 28, 2024 | 9 min read
Table of Contents
  • Pre-requisites