Hello,
I’m trying to switch from the blocking java driver to the new driver with the reactive interface, but I’m facing some serious performance bottleneck.
While migrating our application to spring reactor (using spring data mongodb), I noticed poor performances that could be more than half the existing, so I ran some tests and ended-up with the following: github/Tiller/mongo-perfs (sorry, I can only put 2 links in my post. Direct link in the table below)
Setting aside the spring-data overhead, just by focusing on the blocking driver result vs the reactive driver result, we see a ~40% decrease in performance.
I’m trying to repro your results, but unfortunately when I run ./gradlew test in either project, no tests seem to run. Any idea what the issue might be? I’ve never seen this before. How did you execute the tests?
I did a little digging into your benchmarks, and there is a significant difference between the two. The synchronous benchmark spawns 12 threads and executes 300K operations across those threads. The reactive benchmark, in contrast, doesn’t attempt to limit concurrency at all. Rather, it executes all 300K operations concurrently, relying on the driver to limit concurrency at a lower level due to the max size of the connection pool (which is 100 by default). But even with the driver hitting the breaks in the connection pool, spawning 300K operations concurrently is going to put a lot of memory pressure on the garbage collector (in fact, when I run the benchmarks with default JVM settings, I run out of memory).
Can you try changing your benchmark to provide the same concurrency limits in the reactive benchmark? This can easily be done by creating a Semaphore with NB_THREADS permits. When I tried that, the benchmarks were much closer to each other. Even with that change, the reactive benchmark was still a bit slower, but in my experience that’s in line with expectations.
Releasing the semaphore in the onSubscribe method won’t have the desired effect. Can you try releasing it in the onComplete method of the Subscriber instead?
I am not really understand why limit the number of thread in the reactive driver can improve the performance, all the point of reactive programs tend to work with just a few threads