How OR operator works in Atlas search?

MongoDB Atlas Atlas Search
#1

I am trying to search the document as OR operator which is Should in Mongodb atlas search. but it is really really slow as it seems compared to the must operators. I am using must operator which is really fast and works well. but in case of single search or in SHould operator it will be as slow as 20X as compared to must

am I missing anything?

here is my document sample where I am trying to search - indigenize or technologies,
I also need to search keyword1 or keyword2 or keyword3…or in the future!

Schema-

{
  "_id": {
    "$oid": "63876f3ad75881cafe41a3e9"
  },
  "articleid": "b89bfa05-70b3-11ed-b775-2c59e5044e7b",
  "headline": "Innovative Lessons for Rest of the World",
  "subtitle": "",
  "fulltext": "While the world wants to indigenize high-tech, weuses simple, local technologies to solve most of the problems.",
  "pubdate": "2022-12-01",
  "article_type": "print", 
  "date": 2022-12-01T00:00:00.000+00:00
  }
} 

    [
    {
        "$search":{
            "index":"fulltext",
            "compound":{
                "filter":[
                    {
                        "range":{
                            "path":"date",
                            "gte":"2023-01-30T00:00:00.000Z",
                            "lte":"2023-02-05T00:00:00.000Z"
                        }
                    }
                ],
                "should":[
                    {
                        "text":{
                            "query":"indigenize",
                            "path":[
                                "headline",
                                "fulltext",
                                "subtitle"
                            ]
                        }
                    },
                    {
                        "text":{
                            "query":"technologies",
                            "path":[
                                "headline",
                                "fulltext",
                                "subtitle"
                            ]
                        }
                    }
                ]
            }
        }
    }
]
#2

The performance of Atlas Search’s OR operator (or “should” clause) may slow down when searching for multiple keywords because the engine must retrieve and score documents that match any of the keywords. However, there are a few ways you can improve the performance of the OR operator:

  1. Use a smaller data set: If you are using a large data set, you may want to consider using a smaller data set, which will reduce the amount of data that needs to be processed and improve the overall performance.
  2. Optimize your indexing: Make sure your indexing is optimized for the fields you are searching. If the fields you are searching are not properly indexed, the engine will have to scan the entire collection to find matching documents, which can slow down the performance.
  3. Limit the number of keywords: If you are searching for multiple keywords, try limiting the number of keywords you are searching for to improve performance.
  4. Use a custom scoring function: Atlas Search provides the ability to use a custom scoring function, which allows you to define how the engine scores documents. This can help you prioritize the most relevant results, which can improve the performance of the OR operator.

Here’s an example of how you can optimize your Atlas Search query with a custom scoring function:

[
    {
        "$search": {
            "index": "fulltext",
            "compound": {
                "filter": [
                    {
                        "range": {
                            "path": "date",
                            "gte": "2023-01-30T00:00:00.000Z",
                            "lte": "2023-02-05T00:00:00.000Z"
                        }
                    }
                ],
                "should": [
                    {
                        "text": {
                            "query": "indigenize",
                            "path": [
                                "headline",
                                "fulltext",
                                "subtitle"
                            ],
                            "boost": 2
                        }
                    },
                    {
                        "text": {
                            "query": "technologies",
                            "path": [
                                "headline",
                                "fulltext",
                                "subtitle"
                            ],
                            "boost": 1
                        }
                    }
                ],
                "minimumShouldMatch": "50%"
            }
        }
    }
]

In this example, we’re using a custom scoring function to boost the scores of documents that contain the keyword “indigenize” by a factor of 2. This will help prioritize the most relevant results and improve the performance of the OR operator. Additionally, we’re using the “minimumShouldMatch” parameter to only return documents that match at least one of the keywords.

1 Like
#3

thank you so much for this information, but may I know why a single search is slow too, does it works the same as should Operator? could you please share syntax with the above schema for a single search using date lte, gte and query ?