Autocomplete with filter compound query

I want to search autocomplete on different fields but filtered by a different field

exports.userNameCitySearchAutocomplete = async function (req, res) {

  try {

    const { userNameCityQueryparam } = req.query;

    console.log("search query param", userNameCityQueryparam);

    const agg = [

      {

        $search: {

            index: 'userNameCity',

            'compound': {

             

              "filter": [{

               "text": {

                  "query": ["6271f2bb79cd80194c81f631","62cf35ce62effdc429efa9b0"],

                  "path": "_id",

                }

              }],

            "should": [

              {

              //search on user name

     

              autocomplete: {

                query: userNameCityQueryparam,

                path: 'name',

                fuzzy: {

                  maxEdits: 2,

                  prefixLength: 3

                }

              }},

              //search on user city

           

              {

                autocomplete: {

                query: userNameCityQueryparam,

                path: 'city',

                fuzzy: {

                  maxEdits: 2,

                  prefixLength: 3

                }

              },

           }

           ,

              //search on user contact first name

           

              {

                autocomplete: {

                query: userNameCityQueryparam,

                path: 'contactfirstname',

                fuzzy: {

                  maxEdits: 2,

                  prefixLength: 3

                }

              },

           }

           ,

              //search on user contact last name

           

              {

                autocomplete: {

                query: userNameCityQueryparam,

                path: 'contactlastname',

                fuzzy: {

                  maxEdits: 2,

                  prefixLength: 3

                }

              },

           }

           

          ],

            "minimumShouldMatch": 1

          }

        }

      }

    ]

    const response = await User.aggregate(agg);

    return res.json(response);

    // res.send(response);

  } catch (error) {

    console.log("autocomplete search error", error);

    return res.json([]);

  }

};

===========
Index

{
  "mappings": {
    "dynamic": false,
    "fields": {
      "_id": {
        "type": "string"
      },
      "city": {
        "type": "autocomplete"
      },
      "contactfirstname": {
        "type": "autocomplete"
      },
      "contactlastname": {
        "type": "autocomplete"
      },
      "name": {
        "type": "autocomplete"
      }
    }
  }
}

Issue: am getting empty array when applied filter , if removed filter clause autocomplete search is working

Hi @Manoranjan_Bhol,

Thanks for providing the aggregation pipeline and search index details.

Are you able to provide a few sample documents as well as showing the expected output?

I would like to see how the "_id" values just to verify some theories I have whilst also attempting to re-produce the same behaviour.

In the mean time, if it suits your environment / use case(s), would using the pipeline without the filter whilst using a $match for the "_id" values work for you as a temporary workaround whilst trying to figure out what might be causing the issue you’re experiencing with filter usage? There’s an example of this here.

Regards,
Jason

1 Like

Based off a quick test on my test environment, I was able to create a similar working search aggregation using the compound operator with use of filter and autocomplete.

Sample documents from my test environment:

db.collection.find()
[
  {
    _id: '62e9b5ee86a51031fba35a17',
    title: 'JavaScript beginner course',
    url: 'https://youtube.com/xyz'
  },
  {
    _id: '62e9b5ee86a51031fba35a18',
    title: 'Java beginner course',
    url: 'https://youtube.com/xyz0'
  },
  {
    _id: '62e9b5ee86a51031fba35a19',
    title: 'JavaScript',
    url: 'https://youtube.com/123'
  },
  {
    _id: '62e9b5ee86a51031fba35a20',
    title: 'beginner course - JavaScript',
    url: 'https://youtube.com/xyz123'
  }
]

Note: "_id" is of string type

Search index details:

{
  "mappings": {
    "dynamic": false,
    "fields": {
      "_id": {
        "type": "string"
      },
      "title": {
        "type": "autocomplete"
      }
    }
  }
}

Search aggregation operation and output:

db.collection.aggregate({
  '$search': {
    index: 'default',
    compound: {
      should: [ { autocomplete: { query: 'Java', path: 'title' } } ],
      filter: [
        {
          text: {
            query: [ '62e9b5ee86a51031fba35a17', '62e9b5ee86a51031fba35a18' ],
            path: '_id'
          }
        }
      ]
    }
  }
})
[
  {
    _id: '62e9b5ee86a51031fba35a17',
    title: 'JavaScript beginner course',
    url: 'https://youtube.com/xyz'
  },
  {
    _id: '62e9b5ee86a51031fba35a18',
    title: 'Java beginner course',
    url: 'https://youtube.com/xyz0'
  }
]

Hopefully this can help in some manner. However, if not, please provide the sample documents and expected output :slight_smile:

Regards,
Jason

Document Collection details:

Index details:



Thanks @Jason_Tran . I tried using your suggestion however still did not get any result
Scenario 1: Applied filter by _id—> did not get result i.e. got empty array

Scenario 2: Did not apply filter by _id—> got result i.e. got actual array


Thanks @Manoranjan_Bhol,

I tried using your suggestion however still did not get any result
Scenario 1: Applied filter by _id—> did not get result i.e. got empty array

I presumed based off the search index details you provided for "_id", that it was of string type. In my example, the "_id" is a string but based off your images it appears the "_id" is of ObjectId type.

Would the $match stage suggestion work for you instead? I.e, perform the $search without the filter clause then perform a $match using $in for the object id’s (as there appears to be a few object ID’s you’re attempting to filter for).

Can you provide me 3-5 sample documents as well as the expected / desired output from those 3-5 sample documents?

Regards,
Jason

1 Like

@Marcus
Per your suggestion, posted this issue here
Would you be able to help please

{
index: ‘userNameCity’,
compound:{
should:{

equals: {
value: ObjectId(“6271f2bb79cd80194c81f631”),
path: ‘_id’
},
autocomplete: {
query: ‘tamp’,
path: ‘name’
}
}

}

@Jason_Tran
I checked mongodb documentation.
Should apply equal operator to filter objectId type
However am not able to get the compound operator work
{
index: ‘userNameCity’,
compound:{
should:{

equals: {
value: ObjectId(“6271f2bb79cd80194c81f631”),
path: ‘_id’
},
autocomplete: {
query: ‘tamp’,
path: ‘name’
}
}

}

Can you try changing your index definition, more specifically the "_id" data type? From your initial post (from the index definition), it is stated the “_id” is of data type "string":

{
  "mappings": {
    "dynamic": false,
    "fields": {
      "_id": {
        "type": "string"
      }
...

I believe this needs to be objectId. Please refer to my example below from my test environment (the 2 same aggregations but the index definition field mapping changed for "_id").

Using search index definition where field mapping for "_id" is data type string:

DB>db.collection.aggregate({
  '$search': {
    index: 'default',
    compound: {
      should: [ { autocomplete: { query: 'Java', path: 'title' } } ],
      filter: [
        {
          equals: {
            value: ObjectId("62eb060bbae0364f8a222e72"),
            path: '_id'
          }
        }
      ]
    }
  }
})
/// <--- /// Nothing returned
DB>  

Using search index definition where field mappign for "_id" is data type objectId:

DB>db.collection.aggregate({
  '$search': {
    index: 'default',
    compound: {
      should: [ { autocomplete: { query: 'Java', path: 'title' } } ],
      filter: [
        {
          equals: {
            value: ObjectId("62eb060bbae0364f8a222e72"),
            path: '_id'
          }
        }
      ]
    }
  }
})
/// Output document:
[
  {
    _id: ObjectId("62eb060bbae0364f8a222e72"),
    title: 'JavaScript beginner course',
    url: 'https://youtube.com/xyz'
  }
]
DB>

However am not able to get the compound operator work

If you’re still having issues, please send the following:

  • Any errors being generated
  • The current output
  • 3-4 sample documents
  • The expected output document(s)

Regards,
Jason

@Jason_Tran , Thank you a ton . Finally your solution works. I was able to filter by single objectiId using your solution.

Would you be able to guide me how to filter by array of objectId / multiple objectiId.

The array will have the list of objectId dynamically from the frontend.

Tried with following however does not work.
filter: [
{
equals: {
value: [ObjectId(“62eb060bbae0364f8a222e72”),ObjectId(“62cf35ce62effdc429efa9b0”)]
path: ‘_id’
}

1 Like

Thanks for confirming the solution to the post. I can see you’ve raised another post about this where this is currently ongoing discussion. I will marked this post as close as the original question has been answered.

In terms of filtering for the array of object ID’s, I have not tested on my own environment yet but one alternative, if it suits your use case, would be to run the $search without the object ID filter and then performing a $match stage after to filter on for an array of object ID’s.

A quick example would be something like:

db.collection.aggregate({
  '$search': {
    index: 'default',
    compound: {
      should: [ { autocomplete: { query: 'Java', path: 'title' } } ]
      }
   }
},
{
  '$match': {
    _id: {
      '$in': [ 
        ObjectId("62eb060bbae0364f8a222e72"),
        ObjectId("62eb060bbae0364f8a222e73")
      ] /// <---- Array of Object ID's.
    }
  }
})

This gives the following output:

[
  {
    _id: ObjectId("62eb060bbae0364f8a222e72"),
    title: 'JavaScript beginner course',
    url: 'https://youtube.com/xyz'
  },
  {
    _id: ObjectId("62eb060bbae0364f8a222e73"),
    title: 'Java beginner course',
    url: 'https://youtube.com/xyz0'
  }
]

I’ve not tested this thoroughly but hopefully this helps.

Regards,
Jason

@Jason_Tran
Thank you so much for the help

1 Like

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.