Currently at my company we are using MongoDB as our primary data storage.So I have a collection that maps all our data regarding items and an item can be anything available for sale on shop. So I am using Pagination to fetch the data from database but whenever I use the sort option with a field createdAt it is giving me duplicate entries along with that?
I am using Mongoose ORM on Node JS.
Here is a sample query:
mycollection.find({}, { sort: { createdAt: 1 }, skip: 110, limit: 10 })
That sounds odd, are you sure that duplicates don’t exist and it’s the sorting that’s highlighting the fact you have duplicates?
Yes, I have checked if duplicates exist and also when I remove the sort option there is no duplication.
Can you pick the IDs of two items that are duplicates and look those up directly on the collection and verify what their values are?
There are no duplicates in database. Only while returning results its repeating the documents. I actually found a solution for this- mongodb - Mongo DB duplication issue while using sorting with limit and skip in aggregation - Stack Overflow
When we sort a field that consist of repeating values we need to specify a unique field along with that.So I had to add _id:1 along with the my sort fields.
Ahhh, so it was duplicating over multiple pages of data as the sort was not unique?
You may also want to look at avoiding using .skip and add a criteria that’s indexed, when you get a lot of data and skipping many many pages it can get slow.
Anyway, glad you got it sorted!
Yes Sure I’ll do that too. Thanks for your help.