Atlas search with pagination slow

We have 600,000 documents in our atlas collection, soon to be over 2M. Once we created the proper indexes, an atlas search with" text" or “autocomplete” returns in less than a second. We need pagination so we tried mongoose library mongoose-paginate-v2, it takes 15 seconds!

Any suggestions?

Out of curiosity, have you tried using the $skip aggregation stage?

This can help you for pagination logic fast with $skip and $limit, just convert it to JS or NodeJS :stuck_out_tongue:

def idlimit(page_size, last_id=None, query=None, collection=None):
"""Function returns `page_size` number of documents after last_id
and the new last_id.

if query is None:
    query = {}

if last_id is None:
    # When it is first page
    cursor = collection.find(query).limit(page_size)
    query['_id'] = {'$gt': last_id}
    cursor = collection.find(query).limit(page_size)

# Get the data
data = [x for x in cursor]

if not data:
    # No documents left
    del cursor
    return None, None

# Since documents are naturally ordered with _id, last document will
# have max id.
last_id = data[-1]['_id']

# Return data and last_id
del cursor
return data, last_id

Hey jonathan, I wound up just using atlas search with limit and skip. I implemented virtual scrolling on my vue client and it works really well. Thanks for the response


@Fred_Kufner That’s great to hear. Let us know if you have any more questions.

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.