I’m running MongoDB Community on a machine equipped with a single AMD EPYC 7k62 processor, 128GB of DDR4 3200MHz memory, and a 2TB SSD. The SSD is in good condition, but due to motherboard limitations, it’s capped at PCIe 3.0 x2 bandwidth, providing a maximum throughput of 1700MB/s.
I’m working with a collection that contains approximately 30 million documents, totaling around 400GB in size. The issue I’m facing is that when I use multiple threads to perform concurrent queries on this collection, the disk IO only reaches 150-300 MB/s. This has become a significant bottleneck, slowing down my work considerably, while the CPU remains underutilized.
I’m not an expert in SSD or storage performance, but from what I understand, SSDs can achieve sequential read/write speeds of thousands of megabytes per second (or even up to ~14,000MB/s with the latest PCIe 5.0 SSDs). However, this bandwidth drops significantly when dealing with smaller chunks, such as 4k reads/writes.
This problem occurs on a Windows machine. Unfortunately, due to Windows limitations, I can’t use tools like fio to test disk performance under specific queue depths and thread counts. To work around this, I tested another SSD on a Linux machine using fio. With a chunk size of 4k, a queue depth of 1, and 16 threads, the total bandwidth was around 300MB/s, which is similar to the performance I’m seeing on the Windows machine with MongoDB queries under high thread concurrency. I also used Resource Monitor on Windows to check the average queue depth of the physical disk. It turns out that all these threads and concurrent MongoDB queries only utilize an average queue depth of 0.7.
To summarize, these experiments confirm that MongoDB queries are only utilizing a queue depth of 1 on the SSD, which severely limits performance. My question is: Is there any way to increase the queue depth when running MongoDB queries? Any advice or suggestions would be greatly appreciated!
PS: I’ve never encountered this issue before because I’ve never worked with a single collection large enough to exceed available memory.