we have mongodb with 1 primary and 2 secondaries. On primary node the disk write iops are spiking every minute and latencies go up.
Looking further write iops seems like a bottle neck. so we reduced the fsyc time to 10 sec from 60 sec (default).
It has helped us a lot but still the write iops on 60 qps update are around 1300 in every fsync.
I did a test on another mongodb instance, on 1300 qps 363 iop we used during fsync (every 30 sec).
the major difference between 2 mongos is the db size. IOPS in my mongo seems like an exceptional.
I want to understand how db size impacts write iops during fsync. (In my understanding it should not have).
What other factors are there in fsync write iops?
what data should i be looking at? we dont have any other indexing than _id