Write query blocking other query, performance issue

Hi,

We have a use case where we write in a timeseries (in one query, with insertMany) ~500 000 Documents.

The issue is that this query block mongodb for few second. And all other query are not process in the mean time.

We don’t know how to adress this performance issue. Any help would be welcome :slightly_smiling_face:.

One way I can think of is to throttle the insertion by splitting the 1 insertMany of 500_000 documents into many insertMany of fewer documents. For example, 5 insertMany of 100_000 documents with a small delay between each. You may try to adjust the delay to match the blocking time of the 1 insertMany. For example, if 1 insertMany of 500_000 blocks the server for 5 seconds, wait 1 secs between each of the 5 insertMany. This way other work can be perform in between.

But like any performance issue, the operation you perform is one factor but it also depends on the system characteristics. Because you may throttle as you wish but if you do not have the resources to handle the work load, there is nothing you can do.

1 Like

This is a great workaround, and i will implement it to see how well it works, thanks for the idea :slight_smile:

About system characteristics, we don’t see particular ressource usage when mongodb ingest the 500k documents. We tested on a server (docker + replicaset) and local instance of mongodb (no docker, no replicaset). Would you have an idea where the bottleneck is comming from ?

I have no idea as there is not enough information.