Why PyMongo aggregate is taking too much time?

MongoDB Atlas Atlas Search
, ,
1

Why PyMongo aggregate is taking too much time?

When I use aggregate in pymongo it’s take approx 4-5 seconds.
But when I use Mongodb Atlas UI to query same thing it’s take only 0.01 - 0.07 seconds.

Why my python aggregate query is too slow.

My Analyzed Query Performance

{'stages': [{'$_internalSearchMongotRemote': {'mongotQuery': {'index': 'default', 'text': {'query': 'knowivate developers', 'path': {'wildcard': '*'}}}, 'explain': {'type': 'BooleanQuery', 'args': {'must': [], 'mustNot': [], 'should': [{'type': 'TermQuery', 'args': {'path': 'title', 'value': 'developers'}}, {'type': 'TermQuery', 'args': {'path': 'website', 'value': 'developers'}}, {'type': 'TermQuery', 'args': {'path': 'domain', 'value': 'knowivate'}}, {'type': 'TermQuery', 'args': {'path': 'url', 'value': 'developers'}}, {'type': 'TermQuery', 'args': {'path': 'website', 'value': 'knowivate'}}, {'type': 'TermQuery', 'args': {'path': 'meta_description', 'value': 'developers'}}, {'type': 'TermQuery', 'args': {'path': 'p', 'value': 'knowivate'}}, {'type': 'TermQuery', 'args': {'path': 'url', 'value': 'knowivate'}}, {'type': 'TermQuery', 'args': {'path': 'meta_description', 'value': 'knowivate'}}, {'type': 'TermQuery', 'args': {'path': 'title', 'value': 'knowivate'}}, {'type': 'TermQuery', 'args': {'path': 'domain', 'value': 'developers'}}, {'type': 'TermQuery', 'args': {'path': 'p', 'value': 'developers'}}], 'filter': [], 'minimumShouldMatch': 0}}}}, {'$_internalSearchIdLookup': {}}, {'$limit': 10}, {'$project': {'url': True, 'title': True, 'type': True, 'website': True, 'meta_description': True, '_id': False}}], 'serverInfo': {'host': 'mtm-aws-apsoutheast1-0-m0-18-shard-00-01-c67kj.mongodb.net', 'port': 27000, 'version': '4.4.10', 'gitVersion': '58971da1ef93435a9f62bf4708a81713def6e88c'}, 'ok': 1.0, '$clusterTime': {'clusterTime': Timestamp(1637762646, 4), 'signature': {'hash': b'\xee\x32g\t:(QI\x9e%i\xdd\xc1k+\xa2\xc\xa!\r', 'keyId': 6993066431563064065}}, 'operationTime': Timestamp(1637762646, 4)}

PS

  1. I created index using Mongodb UI > Search Indexes
  2. When I view Indexes there are no indexes.
2

edit:1

I have no option to edit so I am replying.
My query:

query = [{
        '$search': {
        'index': 'default',
        'text': {
            'query': q,
            'path': {
            'wildcard': '*'
        }}}},
        {"$limit" : 10
}]
3

Thanks @Kheersagar_patel I think I know the issue here. This aggregation is likely slower because it is querying a wildcard path rather than a defined one. It should not take quite this long, though. Is there a way to specify a path here? Also, what size collection are you using? How many documents, and what is the size of the index?

2 Likes
4

@Marcus

COLLECTION SIZE: 13.16MB
TOTAL DOCUMENTS: 33591
INDEXES TOTAL SIZE: 2.63MB

How to specify
I have title, description, paragraph, website, url field and I want to use
all of them to query

but,

I have to use title, description and paragraph,

if possible How to use path?

5

Making each of these paths should clauses in a compound operator would be better but this will work as a sanity check for why your queries are slower:

query = [{
        '$search': {
        'index': 'default',
        'text': {
            'query': q,
            'path': ['title', 'description', 'paragraph', 'website', 'url']
        }}},
        {"$limit" : 10
}]
1 Like
6

thank you @Marcus!

One more question.
can i create index index for title, description, paragraph, website and url like this:
title_description_paragraph_website_url_text
and then

query = [{
        '$search': {
        'index': 'default',
        'text': {
            'query': q,
            'path': ['title_description_paragraph_website_url_text']
        }}},
        {"$limit" : 10
}]

Is this make query more faster or Not?