Search filters with multiple conditions

We are trying to replace an aggregation pipeline in our application code where we have a match stage followed by a regex stage with logic that leverages Atlas Search (if this is a good use case). Ultimately we want to get rid of the regex for the purposes of efficiency.

The conditions we search for are as follows - a fictional JSON is provided below:

  1. Filter by:
  • client ID - this is determined by the application
  • documents where an embedded document is either missing or is present and has an attribute with a specific boolean value
  1. Search by:
  • Characters entered by user in a UI the text box (similar to an auto-complete, but not quite there because they currently have to submit the search via form submission). Minimum of one character.

Sample document:

{
  "metaInfo" : {
    "disabled": "false"
  },
  "clientId": "1234",
  "name": "ACME Inc.",
  "Notes": "Here is some data on XYZ..."
}

In the above example, we would be looking for all documents that have clientId of 1234 and either the ‘metaInfo’ embedded document is null OR the ‘metaInfo’ embedded document is present and has an attribute value of disabled = false.

Our current implementation has a few stages, the first few of which are listed below.
Aggregation pipeline stage 1:

{
  $and: [
    {
      clientId: "1234",
    },
    {
      $or: [
        {
          metaInfo: {
            $exists: false,
          },
        },
        {
          "metaInfo.disabled": false,
        },
      ],
    }
  ]
}

The criteria after the above is user input - right now we’re using regex to search for case-insensitive values in the “name” and “notes” attributes:

Aggregation pipeline stage 2:

{
  $or: [
    {
      "name": {
        $regex: "^.*<input string>.*$",
        $options: "i",
      },
    },
    {
      "notes": {
        $regex: "^.*a<input string>*$",
        $options: "i",
      },
    },
  ],
}

I’m struggling to reproduce this logic using Atlas Search - any suggestions would be helpful.

Ignoring the actual user search terms & looking at a specific data set I’m using for testing purposes, this is where I’m currently at using $search and it’s not returning the correct number of results vs. the original pipeline:

{
  "index": "my-search-index",
  compound: {
    "filter": [
      {
        "compound": {
          "minimumShouldMatch": 1,
          "must": [
            {
              "text": {
                "query": "1234",
                "path": "clientId"
              }
            }
          ],
          "should": [
            {
              "compound": {
                "mustNot": {
                  "exists": {
                    "path": "metaInfo"
                  }
                }
              }
            },
            {
              "compound": {
                "must": {
                  "exists": {
                    "path": "metaInfo"
                  }
                },
                "mustNot": {
                  "equals": {
                    "path": "metaInfo.disabled",
                    "value": true
                  }
                }
              }
            }
          ]
        }
      }
    ]
  }
}

Hi @Seamus_CAREY , can you share your index definition, and a sample document that is incorrectly being returned?