MongoDB Data federation limitation

Hello, I m relatively new to MongoDB.
I am working on an application and I have a certain use case where I have to use mongodb lookups on two different collections stored in two different databases. Since, lookups dont have support for cross databases, I decided to use MongoDB Data Federation.
The query works now using federated instance. I read up more about the limitation of Data federation by reading this article data-federation-limitations.

One of the limitations state that federated instance can only have 60 guaranteed concurrent connections per region and only 30 concurrent queries. So if my application were to be used by thousands of users , would there be a performance issue??
If so, how can I do this task differently?

Hello Manish,

Ben here, I’m the PM for Atlas Data Federation. It’s true we have limits on concurrent queries and concurrent connections, but we can expand these limits depending on your requirements. Additionally we plan on offering more configuration options in the future that would help with this.

That said, I’m curious if you can share more about your specific use case. Is the $lookup across databases a very common query? I wonder if there are more appropriate data modeling options that would alleviate the need for a cross DB $lookup.

Best,
Ben

1 Like

Hey Ben, Thanks for the reply.

yes, the $lookup is a very common query for now. The user has a “favorites” section where they maintain their favorite items. The issue is the item and user collections are being maintained in different databases because of some business constraints. Do you think moving both of them to a same DB would be a better idea? Also, if we were to do so, how would it affect the scalability of the application?

Got it. Can I ask how “long running” your $lookup queries are? Do you expect there to be contention over the 30 concurrent query limit and do you know how much we would need to raise the limit by to support your use case?