Increasing search sensitivity

Hello,

Considering the following search word: OptimismPrime and running the following query:


{
  '$search': {
    index: 'clients',
    text: { query: 'optimismprime', path: {wildcard: "*"}, fuzzy: {} }
  }
}

I can successfully find a document containing the word “Optimism Prime” (be it a name, an address or w/e) and it seems the maximum missing data allowed is optimismpri , around 2 missing letters.

However, I would like the search to start finding results much earlier, when a user is sending opt or opti etc.

Could you kindly tell me please, what would be the correct way to achieve this desired result?

Hi @RENOVATIO,

However, I would like the search to start finding results much earlier, when a user is sending opt or opti etc.

Perhaps the autocomplete operator would suit your use case. Please see below the results from my quick test when querying "opt" and "opti" using the autocomplete operator:

  1. Querying using "opt"
DB> db.collection.aggregate({
  '$search': {
    index: 'default',
    autocomplete: { 
      query: 'opt',
      path: 'text',
      fuzzy: {}
    }
  }
})
[
  { _id: ObjectId("6345e56af85ed6b4e320f5b5"), text: 'OptimismPrime' } /// <--- Returned document(s)
]
  1. Querying using "opti"
DB> db.collection.aggregate({
  '$search': {
    index: 'default',
    autocomplete: { 
      query: 'opti',
      path: 'text',
      fuzzy: {}
    }
  }
})
[
  { _id: ObjectId("6345e56af85ed6b4e320f5b5"), text: 'OptimismPrime' } /// <--- Returned document(s)
]

The above was done using a single test document so it is highly recommend you test thoroughly in your own test environment to verify usage of the autocomplete operator suits your use case and requirements.

Regards,
Jason

1 Like

Hello Jason,

Thank you for your reply and examples. I had considered autocomplete, but there was one apparent problem that held me back from opting for it. Doesn’t autocomplete require only 1 designated path field? Meaning, I have to choose 1 field only to run the search on, for example a “name” field, but then not the “address” or anything else.

For example, if I have a username John Brownfield, on the address Green Tree Street, 20, with autocomplete I would have to choose only 1 field to scan, in this case either username or address.

Is there a way to find John Brownfield’s document, by partially entering just part of his surname, e.g. brown or his address, e.g. only green? The search query is passed through REST api ?search=query param, meaning the system is not aware of which field (I am assuming that’s what the path is for) is targetted.
Thank you again and I apologize if my original request was not as comprehensive about the issue.

1 Like

For example, if I have a username John Brownfield, on the address Green Tree Street, 20, with autocomplete I would have to choose only 1 field to scan, in this case either username or address .

Thanks for the clarification regarding the mutliple fields that need to be searched on. I presume you tested autocomplete with a {"wildcard":"*"} path value but correct me if I am wrong here.

Is there a way to find John Brownfield’s document, by partially entering just part of his surname, e.g. brown or his address, e.g. only green ? The search query is passed through REST api ?search=query param

In saying so, have you tried compound with autocomplete to see if it meets your requirements? I did 2 simple tests based off your example:

  1. Passing through the query value of "John"

  1. Passing through the query value of "Green"

The two tests were to try and retrieve the same document as you have mentioned.

Examples are from Data Explorer

If this is not what you’re exactly after, there is a feedback post regarding autocomplete usage on multiple fields (which sounds like it would suit your use case). It is currently being worked on at this stage.

Regards,
Jason

1 Like

@Jason_Tran thank you very much for the extremely clear and useful examples. To answer your question, yes I have tried using the wildcard, however I received an error from Mongo that autocomplete only accepts a string.

Your solution appears to be very usable and I think that is what I will probably go with. Just 2 last questions if I may.

  1. Are there any considerations, pitfalls or risks (performance, usability or otherwise) to be aware of when going the compound route? Especially if there are more than 2 searchable fields, say for example 3-10?
  2. In order to use autocomplete is it necessary to configure the respective field types as autocomplete on the mongo atlas dashboard? I am asking this, because when testing autocomplete, I seem to have had them working regardless of those fields being created or not in atlas.

Thank you very much, Jason

Hi @RENOVATIO

  1. Are there any considerations, pitfalls or risks (performance, usability or otherwise) to be aware of when going the compound route? Especially if there are more than 2 searchable fields, say for example 3-10?

Not specific to the compound operator but more so the autocomplete operator for your use case, perhaps taking a look at the performance considerations mentioned here will help.

  1. In order to use autocomplete is it necessary to configure the respective field types as autocomplete on the mongo atlas dashboard? I am asking this, because when testing autocomplete, I seem to have had them working regardless of those fields being created or not in atlas.

Just to clarify, do you mean setting index field definition to contain autocomplete for the fields? E.g.:

  1. Setting "username" to data type autocomplete
  2. Setting "address" to data type autocomplete

If you have a screenshot of what you are referring in the atlas dashboard, that would help clarify the question for me just to be sure :slight_smile:

Additionally, regarding your testing mentioned above, I presume you created the index without specifying the data type configuration of autocomplete? Is this correct? I.e. created the search index without specifying autocomplete in the index definition and then running an autocomplete search query which returned your required results?

Regards,
Jason

1 Like

Hello @Jason_Tran, forgive me the lack of clarity.

Yes, that’s right, this is what I intended. I just created a new search index without specifying any fields or their types and the search was working anyway, so I was wondering why was that the case and when then does one need to create those explicit fields.

1 Like

Interesting - Do you have some quick steps that I can reproduce this behaviour? Please include the index details and query used.

Regards,
Jason

1 Like

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.