Support cross field search in MongoDB across all fields asterixis

Working with Data
,
1

I want to support cross field searching in a mongoDB.

So my REST query would look like:

GET /endpoint?query=v5?query_fields="record.k1,record.k5.k51"

Now I am thinking of translating it to a MongoDB find as

my_collection.find({query: {"record.k1":"v5", "record.k5.k51":"v5"}}, {options:{}});

Now I need to support asterix also, something like:
GET /endpoint?query=v5?query_fields="*"

My document can be very very nested.

So how can I achieve this asterix (*) cross field searching in MongoDB JavaScript native driver?

Thanks.
Pradip

2

Hi @Pradip_Speaks,
You can use the Text Index and the $text query.
See this for an example.
Wildcard indexes cannot support queries using the $text operator, so you cannot use this approach to achieve your goal.

A suggestion for solution to your problem is to save the whole record field’s value as a string using JSON.stringify when saving the data and JSON.parse when retrieving the data. That way, you can create a text index on the record’s field.

Alternatively, if text’s index doesn’t fit your task, you can use the $regex operator.

Goodluck,
Rafael,

3

Thanks.

Does $text query poses any security risk (script injection etc.) to be passed inside the APIs?

Many places I was reading people propose to use mongo-sanitizer npm module to sanitize the payload, options, queries etc. That will remove anything with dollar($).

4

It depends on how you’re using the input in your server code.
If you’re using it inside db.runCommand(command) and command is a string, then script injection is an issue.
If you run this for example:

db.collection.findOne({name: nameVariable})

where nameVariable is a string you got from API, then script injection is not an issue.

See this for more information.
There is no NoSql attack in the list but it’s similar to other injection attacks.
Search injection attacks in the list.

Thanks,
Rafael,

1 Like
5

Special thanks for the link to owasp.com.

6

Thanks.

I am going to construct the object of $text option and pass to findOne/find. In that case, do we see any potential security risk of using $text?

7

@Pradip_Speaks, I cannot tell without seeing the code

8

@Rafael_Green,
The code is like this:

if (searchString) {
            crossFieldQueries.push({$text: { $search : searchString }});
}
let allQueries = {};
   ....
allQueries = [...crossFieldQueries];
query = Object.assign({}, ...allQueries);
console.log('Query: ', query);

The query is coming like this:

Query:  { '$text': { '$search': 'v1' } }

Do you see an issue now to pass this query in the find as : collection.find(query, {});

9

looks safe to me
but don’t take my answer as security guarantee

Goodluck,
Rafael,

1 Like
10

Thanks, @Rafael_Green. Appreciate your view and help here.

1 Like
11

@Rafael_Green - I tried out the approach above, same code.

Final Query passed to find: {$text":{"$search":"v1"}}

However I am getting this error: MongoError: $text not supported

What’s the mistake?

Also, if some reason I can not use $text, you suggested $regex.

Using $regex - how can I get the behavior like $text? (i.e. searching a string across all fields of my collection).

[I am using CosmosDB Mongo DB from JavaScript nativeMongo client)].

Thanks,
Pradip

12

Just to be clear: My purpose is not to search for regex star. Rather I want to search a string across all the fields of the documents in my collection.
Basically it’s like:
?query_fields=*&query=search_val

Where query fields are the name of the fields of the document and search_val is the value to be searched.

The $text is showing MongoError: $text not supported.

Any help will be highly solicited.

Thanks,
Pradip

13

Hi @Pradip_Speaks,

Cosmos’ emulated API is not the same product as MongoDB:

Text search (introduced in MongoDB 2.4, March 2013) is not currently supported by Cosmos’ emulated API and apparently is not on their roadmap per Add support for “Search text” ($text Operator).

If full MongoDB feature support is important to your use case, I would consider using MongoDB Atlas on Azure for a managed data service.

Regards,
Stennie

1 Like
14

Thanks @Stennie_X .

Then if I want to search something across the documents (the question discussed in this thread), can you please suggest if I can use regexp someway?

Thanks,
Pradip

15

Hi @Pradip_Speaks,

I’m honestly not sure what is supported or efficient in Cosmos’ API. Unfortunately my knowledge of the MongoDB server doesn’t provide any insight into Cosmos’ implementation or behaviour.

However, I would note that regular expressions are designed for pattern matching, which is a different use case than full text search based on language heuristics (eg stemming and stop words). From what you’ve described so far (searching all fields in highly nested documents), I expect you want a text search implementation rather than regex.

In the Cosmos feedback item I linked in my previous post, an Azure Product Manager suggested Azure Search is their recommended solution.

Regards,
Stennie

1 Like
16

Thanks @Stennie_X . Thanks for all the pointers and the direction. Much appreciated.

1 Like