Erik Hatcher

# The Atlas Search 'cene: Season 1

Welcome to the first season of a video series dedicated to Atlas Search!  This series of videos is designed to guide you through the journey from getting started and understanding the concepts, to advanced techniques.

## What is Atlas Search?

[Atlas Search][1] is an embedded full-text search in MongoDB Atlas that gives you a seamless, scalable experience for building relevance-based app features. Built on Apache Lucene, Atlas Search eliminates the need to run a separate search system alongside your database.

By integrating the database, search engine, and sync mechanism into a single, unified, and fully managed platform, Atlas Search is the fastest and easiest way to build relevance-based search capabilities directly into applications.

> Hip to the *'cene*
> 
> The name of this video series comes from a contraction of "Lucene",
> the search engine library leveraged by Atlas. Or it's a short form of "scene". 

## Episode Guide

### **[Episode 1: What is Atlas Search & Quick Start][2]**

In this first episode of the Atlas Search 'cene, learn what Atlas Search is, and get a quick start introduction to setting up Atlas Search on your data.  Within a few clicks, you can set up a powerful, full-text search index on your Atlas collection data, and leverage the fast, relevant results to your users queries.

:youtube[]{vid=HsS0z3eOCSQ}

### **[Episode 2: Configuration / Development Environment][3]**

In order to best leverage Atlas Search, configuring it for your querying needs leads to success. In this episode, learn how Atlas Search maps your documents to its index, and discover the configuration control you have.

:youtube[]{vid=3fRCD_QptVQ}

### **[Episode 3: Indexing][4]**

While Atlas Search automatically indexes your collections content, it does demand attention to the indexing configuration details in order to match users queries appropriately. This episode covers how Atlas Search builds an inverted index, and the options one must consider.

:youtube[]{vid=XrJG994YxD8}

### **[Episode 4: Searching][5]**

Atlas Search provides a rich set of query operators and relevancy controls. This episode covers the common query operators, their relevancy controls, and ends with coverage of the must-have Query Analytics feature.

:youtube[]{vid=U-hAo_TAi9c}

### **[Episode 5:  Faceting][6]**

Facets produce additional context for search results, providing a list of subsets and counts within. This episode details the faceting options available in Atlas Search.

:youtube[]{vid=90icEZ8fzJE}

### **[Episode 6: Advanced Search Topics][7]**

In this episode, we go through some more advanced search topics including embedded documents, fuzzy search, autocomplete, highlighting, and geospatial.

:youtube[]{vid=Ibdb81WXOxE}

### **[Episode 7:  Query Analytics][8]**

Are your users finding what they are looking for? Are your top queries returning the best results? This episode covers the important topic of query analytics. If you're using search, you need this!

:youtube[]{vid=QIYNsAXPJUA}

### **Episode 8:  Tips & Tricks [Coming Soon!]**

In this final episode of The Atlas Search 'cene Season 1, useful techniques to introspect query details and see the relevancy scoring computation details. Also shown is how to get facets and search results back in one API call.


  [1]: https://www.mongodb.com/atlas/search
  [2]: https://www.mongodb.com/developer/videos/what-is-atlas-search-quick-start/
  [3]: https://www.mongodb.com/developer/videos/atlas-search-configuration-development-environment/
  [4]: https://www.mongodb.com/developer/videos/mastering-indexing-for-perfect-query-matches/
  [5]: https://www.mongodb.com/developer/videos/query-operators-relevancy-controls-for-precision-searches/
  [6]: https://www.mongodb.com/developer/videos/faceting-mastery-unlock-the-full-potential-of-atlas-search-s-contextual-insights/
  [7]: https://www.mongodb.com/developer/videos/atlas-search-mastery-elevate-your-search-with-fuzzy-geospatial-highlighting-hacks/
  [8]: https://www.mongodb.com/developer/videos/atlas-search-query-analytics/

The Atlas Search 'cene: Season 1

We've grown accustomed to expecting the best results for our search intentions. Now it’s your turn to build the same experience into your Atlas-powered app. 

Atlas Search Relevancy Explained

This article delves into using the Atlas Search support built into the MongoDB Java driver

Using Atlas Search from Java

This article details one of my MongoDB Atlas learning journeys. I joined MongoDB in the fall of 2022 as a Developer Advocate for Atlas Search. With a couple of decades of Lucene experience, I know search, but I had little experience with MongoDB itself. As part of my initiation, I needed to learn the [MongoDB Query API](https://www.mongodb.com/mongodb-query-api), and coupled my learning with my Wordle interest.

## Game on: Introduction to Wordle

The online game [Wordle](https://www.nytimes.com/games/wordle/index.html) took the world by storm in 2022. For many, including myself, Wordle has become a part of the daily routine. If you’re not familiar with Wordle, let me first apologize for introducing you to your next favorite time sink. The Wordle word guessing game gives you six chances to guess the five-letter word of the day. After a guess, each letter of the guessed word is marked with clues indicating how well it matches the answer. Let’s jump right into an example, with our first guess being the word `ZESTY`. Wordle gives us these hints after that guess:

![ZESTY](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/guess1_7613a12958.png)

The hints tell us that the letter E is in the goal word though not in the second position and that the letters `Z`, `S`, `T`, and `Y` are not in the solution in any position. Our next guess factors in these clues, giving us more information about the answer:

![ZESTY BREAD](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/guess2_ba233dfa9c.png)

Do you know the answer at this point? Before we reveal it, let’s learn some MongoDB and build a tool to help us choose possible solutions given the hints we know.

## Modeling the data as BSON in MongoDB

We can easily use the forever free tier of hosted MongoDB, called Atlas. To play along, visit the [Atlas homepage](https://www.mongodb.com/atlas/database) and create a new account or log in to your existing one.

Once you have an Atlas account, create a database to contain a collection of words. All of the possible words that can be guessed or used as daily answers are built into the source code of the single page Wordle app itself. These words have been extracted into a list that we can quickly ingest into our Atlas collection.

I created a repository for the data and code [here](https://github.com/mongodb-developer/mongordle). The README shows how to import the word list into your Atlas collection so you can play along.

The query operations needed are:

* Find words that have a specific letter in an exact position.
* Find words that do not contain any of a set of letters.
* Find words that contain a set of specified letters, but not in any known positions.

In order to accommodate these types of criteria, a word document looks like this, using the word MONGO to illustrate:

```
{
  "_id":"MONGO",
  "letter1":"M",
  "letter2":"O",
  "letter3":"N",
  "letter4":"G",
  "letter5":"O",
  "letters":["M","O","N","G"]
}
```

## Finding the matches with the MongoDB Query API

Each word is its own document and structured to facilitate the types of queries needed. I come from a background of full-text search where it makes sense to break down documents into the atomic findable units for clean query-ability and performance. There are, no doubt, other ways to implement the document structure and query patterns for this challenge, but bear with me while we learn how to use MongoDB Query API with this particular structure. Each letter position of the word has its own field, so we can query for exact matches. There is also a catch-all field containing an array of all unique characters in the word so queries do not have to be necessarily concerned with positions.

Let’s build up the MongoDB Query API to find words that match the hints from our initial guess. First, what words do not contain `Z`, `S`, `T`, or `Y`? Using MongoDB Query API [query operators](https://www.mongodb.com/docs/manual/reference/operator/query/) in a `.find()` API call, we can use the [`$nin` $not in$ operator](https://www.mongodb.com/docs/manual/reference/operator/query/nin/) as follows:

```
{
  "letters":{
    "$nin":["Z","S","T","Y"]
  }
}
```
Independently, a `.find()` for all words that have a letter `E` but not in the second position looks like this, using the [`$all` operator](https://www.mongodb.com/docs/manual/reference/operator/query/all/) as there could be potentially multiple letters we know are in the solution but not which position they are in:

```
{
  "letters":{
    "$all":["E"]
  },
  "letter2":{"$nin":["E"]}
}
```

To find the possible solutions, we combine all criteria for all the hints. After our `ZESTY` guess, the full `.find()` criteria is:

```
{
  "letters":{
    "$nin":["Z","S","T","Y"],
    "$all":["E"]
  },
  "letter2":{"$nin":["E"]}
}
```

Out of the universe of all 2,309 words, there are 394 words possible after our first guess.

Now on to our second guess, `BREAD`, which gave us several other tidbits of information about the answer. We now know that the answer also does not contain the letters `B` or `D`, so we add that to our letters field `$nin` clause. We also know the answer has an `R` and `A` somewhere, but not in the positions we initially guessed. And we have now know the third letter is an `E`, which is matched using the [`$eq` operator](https://www.mongodb.com/docs/manual/reference/operator/query/eq/). Combining all of this information from both of our guesses, `ZESTY` and `BREAD`, we end up with this criteria:

```
{
  "letters":{
    "$nin":["Z","S","T","Y","B","D"],
    "$all":["E","R","A"]
  },
  "letter2":{"$nin":["E","R"]},
  "letter3":{"$eq":"E"},
  "letter4":{"$nin":["A"]}
}
```

Has the answer revealed itself yet to you? If not, go ahead and import the word list into your Atlas cluster and run the aggregation.

It’s tedious to accumulate all of the hints into `.find()` criteria manually, and duplicate letters in the answer can present a challenge when translating the color-coded hints to MongoDB Query API, so I wrote a bit of Ruby code to handle the details. From the command-line, using [this code](https://github.com/mongodb-developer/mongordle), the possible words after our first guess looks like this….

```
$ ruby word_guesser.rb "ZESTY x~xxx"
{"letters":{"$nin":["Z","S","T","Y"],"$all":["E"]},"letter2":{"$nin":["E"]}}
ABIDE
ABLED
ABODE
ABOVE
.
.
.
WOVEN
WREAK
WRECK
394
```

The output of running `word_guesser.rb` consists first of the MongoDB Query API generated, followed by all of the possible matching words given the hints provided, ending with the number of words listed. The command-line arguments to the word guessing script are one or more quoted strings consisting of the guessed word and a representation of the hints provided from that word where `x` is a greyed out letter, `~` is a yellow letter, and `^` is a green letter. It’s up to the human solver to pick one of the listed words to try for the next guess. After our second guess, the command and output are:

```
$ ruby word_guesser.rb "ZESTY x~xxx" "BREAD x~^~x"
{"letters":{"$nin":["Z","S","T","Y","B","D"],"$all":["E","R","A"]},"letter2":{"$nin":["E","R"]},"letter3":{"$eq":"E"},"letter4":{"$nin":["A"]}}
OPERA
1
```

Voila, solved! Only one possible word after our second guess.

![OPERA](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/Screen_Shot_2023_02_01_at_4_21_05_PM_4c7085b7c5.png)

In summary, this fun exercise allowed me to learn MongoDB’s Query API operators, specifically `$all`, `$eq`, and `$nin` operators for this challenge.

To learn more about the MongoDB Query API, check out these resources:
* [Introduction to MongoDB Query API](https://www.mongodb.com/presentations/an-introduction-to-mongodb-query-api-webinar-recording-na)
* [Getting Started with Atlas and the MongoDB Query Language $MQL$](https://www.mongodb.com/developer/products/atlas/getting-started-atlas-mongodb-query-language-mql/)(now referred to as the MongoDB Query API)
* The free [MongoDB CRUD Operations: Insert and Find Documents course](https://learn.mongodb.com/courses/mongodb-crud-operations-insert-and-find-documents) at MongoDB University

Let’s learn a few MongoDB Query API operators while solving Wordle

Erik Hatcher

Content by Erik Hatcher

The Atlas Search 'cene: Season 1

Atlas Search Relevancy Explained

Using Atlas Search from Java

Wordle Solving Using MongoDB Query API Operators