MongoDB Developer
Atlas
MongoDB Developer Centerchevron-right
Developer Topicschevron-right
Productschevron-right
Atlaschevron-right

Flexible Querying with Atlas Search

Ethan SteiningerPublished Oct 04, 2022 • Updated Oct 12, 2022
GraphQLAtlasSearchJavaScript
SNIPPET
Copy Link
facebook icontwitter iconlinkedin icon
random alt
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty

Introduction

In this walkthrough, I will show how the flexibility of Atlas Search's inverted indexes are a powerful option versus traditional b-tree indexes when it comes to supporting ad-hoc queries.

What is flexible querying?

Flexible query engines provide the ability to execute a performant query that spans multiple indexes in your data store. This means you can write ad-hoc, dynamically generated queries, where you don't need to know the query, fields, or ordering of fields in advance.
Be sure to check out the MongoDB documentation on this subject!
It's very rare that MongoDB’s query planner selects a plan that involves multiple indexes. In this tutorial, we’ll walk through a scenario in which this becomes a requirement.
Your application is in a constant state of evolution
Let’s say you have a movie application with documents like:
Initial product requirements
Now for the version 1.0 application, you need to query on title and year, so you first create a compound index via:
db.movies.createIndex( { "title": 1, "year": 1 } )
Then issue the query:
db.movies.find({"title":"Fight Club", "year":1999})
When you run an explain plan, you have a perfect query with a 1:1 documents-examined to documents-returned ratio:
Our query then needs to evolve
Now our application requirements have evolved and you need to query on cast and imdb. First you create the index:
db.movies.createIndex( { "cast": 1, "imdb.rating": 1 } )
Then issue the query:
db.movies.find({"cast":"Edward Norton", "imdb.rating":{ $gte:9 } })
Not the greatest documents-examined to documents-returned ratio, but still not terrible:
Now our query evolves again
Now, our application requires you issue a new query, which becomes a subset of the original:
db.movies.find({"imdb.rating" : { $gte:9 } })
The query above results in the dreaded collection scan despite the previous compound index (cast_imdb.rating) comprising the above query’s key. This is because the "imdb.rating" field is not the index-prefix, and the query contains no filter conditions on the "cast" field."
Note: Collection scans should be avoided because not only do they instruct the cursor to look at every document in the collection which is slow, but it also forces documents out of memory resulting in increased I/O pressure.
Our query plan results as follows:
Now you certainly could create a new index composed of just imdb.rating, which would return an index scan for the above query, but that’s three different indexes that the query planner would have to navigate in order to select the most performant response.
Because Lucene uses a different index data structure (inverted indexes vs B-tree indexes), it’s purpose-built to run queries that overlap into multiple indexes.
Unlike compound indexes, the order of fields in the Atlas Search index definition is not important. Fields can be defined in any order. Therefore, it's not subject to the limitation above where a query that is only on a non-prefix field of a compound index cannot use the index.
If you create a single index that maps all of our four fields above (title, year, cast, imdb):
Then you issue a query that first spans title and year via a must (AND) clause, which is the equivalent of db.collection.find({"title":"Fight Club", "year":1999}):
The corresponding query planner results:
Then when you add imdb and cast to the query, you can still get performant results:
The corresponding query planner results:

This isn’t a peculiar scenario

Applications evolve as our users’ expectations and requirements do. In order to support your applications' evolving requirements, Standard B-tree indexes simply cannot evolve at the rate that an inverted index can.
Use cases
Here are several examples where Atlas Search's inverted index data structures can come in handy, with links to reference material:
  • GraphQL: If your database's entry point is GraphQL, where the queries are defined by the client, then you're a perfect candidate for inverted indexes
  • Advanced Search: You need to expand the filtering criteria for your searchbar beyond several fields.
  • Wildcard Search: Searching across fields that match combinations of characters and wildcards.
  • Ad-Hoc Querying: The need to dynamically generate queries on-demand by our clients.

Resources


Copy Link
facebook icontwitter iconlinkedin icon
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty
Related
Article

Auto Pausing Inactive Clusters


Nov 03, 2022
Tutorial

Influence Search Result Ranking with Function Scores in Atlas Search


May 16, 2022
Tutorial

Implement Update and Delete Operations in React Using the Atlas GraphQL API


Nov 01, 2022
Tutorial

Write A Serverless Function with AWS Lambda and MongoDB


Sep 23, 2022
Table of Contents