Hey,
I am using “$in” operator to pass ids in mogodb query. The problem is the number of ids can be anywhere between 1 to 1k.
What is the limit to use $in operator?
Documentation does not suggest that things will go well doing that:
The recommendation there is in the 10’s of items, more than that can cause performance issues.
What exactly are you trying to do with a find using “$in” this large?
Hey @John_Sewell
Thank you for the response, In my project I have a requirement such that there are number of providers and I need to get the providers which are available. I don’t want to get all the providers.
Thank you!
How many are there, at some point you’re better doing not equal to!
Are there no other options to filter on? In a relational system doing massive In statements can also cause issues (in a coincidence I ran into this today when debugging some code!) so you may join onto another table that can have the filter applied or something similar.
Is it possible to update the data to have a marker or something that’s easier to search for without doing a massive $in?
Sorry I cant think of an immediate quick solution, it kinds of depends on the data you have and what changes you can do. Perhaps someone else can chip in with a solution they’ve deployed before.
1 Like
Thank you @John_Sewell , There is no marker to identify the ids in the collection for the given requirement.
If I do not use “$in” then I have to use lookup.
1 Like
While there is no hard limit on the number of values you can use with the $in operator, practical limitations arise due to the BSON document size restriction. If your $in query results in a BSON document that exceeds the 16 MB limit, MongoDB will reject the query.
2 Likes
True, possibly best to try and take some metrics on how it actually performs with your workload…try it with 10,100,1000,10000 elements in the $in stage and see how performance changes before you do anything drastic!
Thank you @adeola_oladeinde and @John_Sewell , I will try it as you suggested with different limits and check the result. Thank you!
1 Like
This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.