How to reduce the maximum insert time to less than 1s

feng_deng · June 27, 2025, 7:15am

I don’t find any bottlenecks (CPU, memory, Disk IO and network), but when insert requests are approaching 2W/s, some insert requests take >1s, which causes some problems with my service, I want to reduce all insert to less than 1s, any suggestions?

Here’s a list of my server quotas:
3 mongos ecs.hfc6.2xlarge 8c16G
3 confsvr ecs.c6e.xlarge 4c8G
4 * 1 shard [PSA, P/S(ecs.r6.3xlarge, 12c96G) , Arbiter(ecs.t6-c2m1.large/ecs.t5-lc1m2.small, 1c2G/2c1G )]

confsvr :

storage:
  dbPath: /path/to/data/
  journal:
    enabled: true
  directoryPerDB: true
  engine: "wiredTiger"
  wiredTiger:
    engineConfig:
      journalCompressor: "zlib"
      directoryForIndexes: true
    collectionConfig:
      blockCompressor: "zlib"
operationProfiling:
  mode: slowOp
  slowOpThresholdMs: 1000

shardsvr:

storage:
  dbPath: /path/to/data/
  journal:
    enabled: true
    #indexBuildRetry: false
  directoryPerDB: true
  engine: "wiredTiger"
  wiredTiger:
    engineConfig:
      #cacheSizeGB: 80
      journalCompressor: "zlib"
      directoryForIndexes: true
    collectionConfig:
      blockCompressor: "zlib"
operationProfiling:
  mode: slowOp
  slowOpThresholdMs: 1000

24h monitor data:
1 of 3 Mongos

Confsvr primary:

Data Disk type

Shard1 primary:
Data Disk type

1 Mongos stat

Shard1 primary stat

ops

Alexander_Komyagin1 · July 2, 2025, 8:34am

You need to see what inserts are taking longer than 1 second in the logs and find out why. This could be because of some latency, replication lag between your primary and secondary, or maybe those are batch inserts. MongoDB logs are descriptive enough to help you triage the problem. Look into shard primaries’ logs first.