Hi @MaBeuLux88_xxx
Yes correct on the storage side but the writes are very variable much higher during market hours and tailing off steeply at night, it peaks at ~20K inserts and is rarely under 7k during the day.
I did set up a sharded cluster a couple of years ago but throughput was disappointing, I can revisit it, I may have done something wrong. I wasn’t aware of the 2TB guideline.
TTL indexes we didn’t try yet I thought that might be too hard to manage being time based so a flood of data on a particularly busy day would not get deleted promptly.
Standalone machine, no oplog, when I joined the project we had dial node replication and capped collections but we had to remove replication as the volume grew. If we are at the limits of vertical scaling then it’s probably time to revisit shards but should we really expect such a huge discrepancy between insert and delete?
the machine spec is
$lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 80
On-line CPU(s) list: 0-79
Thread(s) per core: 2
Core(s) per socket: 20
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 79
Model name: Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz
Stepping: 1
CPU MHz: 2200.000
CPU max MHz: 2200.0000
CPU min MHz: 1200.0000
BogoMIPS: 4394.81
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 51200K
NUMA node0 CPU(s): 0-19,40-59
NUMA node1 CPU(s): 20-39,60-79
with 64Gb installed
Cheers