Why is MongoClients.create() function speeding up after repeated function invocattions?

Working with Data Drivers & ODMs
#1

I am working on a demo Java application that uses MongoDB for basic retrievals of data and I am using mongodb-driver-sync:4.4.1 Java driver.

In that, I am trying to analyze which operation of MongoDB is taking how much time, so that I can speed up my application by writing code efficiently. After experimenting, I noticed that MongoClients.create() function is taking about 170ms to execute. After only a few repeated continuous invocations, this time started decreasing to about 130ms. But I performed a brief context switch of applications in the OS (like spending time on browser and other applications). After coming back to the Java application, I noticed that the time taken by the above function increased to around 170ms, which again gradually decreased to about 130ms after repeated execution without any context switch.

This behaviour is affecting the accuracy of my experiments and is causing the total time to vary by +/-40ms because of the MongoClients.create() function. Can anyone please explain why this behaviour is occurring? Is the OS trying to cache something? If yes, what is it trying to do?

OS version: macOS Catalina, v10.15.7

Any help is appreciated. Thanks.

#3

MongoClients.create() should not be part of your part of your performance benchmarks.

You should be calling it only once, in the initialization code of your application, and reuse the result for your queries.

#4

Thank you for replying.

You are correct. I must not include that part for benchmarking. But I am not exactly benchmarking the application. I am executing the program only once, and I am trying to record the “wall clock” time difference a real user might informally experience while executing the whole program. This measurement must include the time for MongoClients.create() as it is also a part of the program execution and the end user faces the delay due to the above function as well. You can assume that this is not a server application that is continuously running.

I only need to know why this particular function is behaving in such a manner. I have observed this behaviour in functions that contain I/O such as reading files. In such cases, the differences can be attributed to the OS caching the file. But what is causing MongoClients.create() to show a considerable difference of 40-45ms?

#5

A lot of things can explain.

Where is mongod server running?

Is it a dedicated server?

Are the client and server running on a private network with isolated traffic? Both running on the same machine?

Do you start and stop your java apps between each invocations? If you don’t then the first invocation is byte code interpretation, the others migth be the compiled version.

There is probably more than 40ms overhead just to start your java applications.

There is propably nothing inherently inside .create to justify that difference. You might even reach some TCP optimization when requesting some remote IP/port from same client.

#6

Where is mongod server running?

It is running locally, on localhost:27017.

Are the client and server running on a private network with isolated traffic? Both running on the same machine?

Both are running on the same machine. There is no external network involved and hence network delay cannot come into light here.

Do you start and stop your java apps between each invocations?

Yes, but only the client is restarted after each invocation. So, the JIT cannot come into play in this case.

And yes, I went through the source code of the driver but not extensively. I too did not find anything that can explain this. Regarding TCP optimizations, what kind of optimizations are generally performed? Because are they capable of providing as high as 40ms difference in connection times?

#7

Is the MongoDB server trying to remember the client metadata when the same client connects?
Does the server skip a few steps if it detects the same client?

Mongod version: 5.0.7

#8

I would be surprise if there is no context switch since

But context switches are not really an issue unless your machine needs paging, when browsing and does not need it when simply running your test.

If I understand correctly, the user of your application will be running both the mongod server and your application on the same machine? If not, are your wall clock tests really pertinent, if your typical user will have a different architecture?

I think you are worrying too much about this 130ms vs 170ms variation you use-case involve starting the whole application. That is what I assume with

What is the average total wall clock of the use-case, from the start of the non-server application and the result a typical user will see? 130ms vs 170ms might be perceptible but 1130ms vs 1170ms might not be so much.

Anyway, what ever optimizations TCP, Firewall or mongod is doing or not, I do not think you can do much.

Things you can do if that 40ms is that important

1 - try to connect direct with (not sure if it works in Windows)
mongodb://%2Ftmp%2Fmongodb-27017.sock
2 - play with pool size parameters, with a non server application, you might be able to reduce the number of connection to a very small value

#9

If I understand correctly, the user of your application will be running both the mongod server and your application on the same machine?

Yes, both need to be running on the same machine.

Anyway, what ever optimizations TCP, Firewall or mongod is doing or not, I do not think you can do much.

True, but at this point I only need to know the exact cause of this behaviour.

The average time is around 500ms of the entire application, so the difference might be noticeable. Anyways, I will try to work on the last two points you have mentioned. Thanks.

1 Like