MongoDB consuming all available memory

[Deepak_Kumar16 (Deepak Kumar)]

@billy_noah

I’m having an issue with MongoDB continually consuming all available RAM and then getting killed by oom. I’ve read a few questions on the stackexchange network that suggest setting storage.wiredTiger.engineConfig.cacheSizeGB can resolve the issue but it is not helping.

Right at this moment, here is the situation:

Mongod Service Status

$ service mongod status
● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/mongod.service.d
             └─always_restart.conf
     Active: active (running) since Mon 2023-03-06 12:57:35 EST; 3min 26s ago
       Docs: https://docs.mongodb.org/manual
   Main PID: 13318 (mongod)
     Memory: 6.4G
     CGroup: /system.slice/mongod.service
             └─13318 /usr/bin/mongod --config /etc/mongod.conf

Mar 06 12:57:35 chat systemd[1]: Stopped MongoDB Database Server.
Mar 06 12:57:35 chat systemd[1]: Started MongoDB Database Server.

Free Memory

$ free -h
              total        used        free      shared  buff/cache   available
Mem:          7.8Gi       7.2Gi       129Mi       4.0Mi       435Mi       309Mi
Swap:         511Mi       511Mi          0B

Mongo Config

# Where and how to store data.
storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true
  engine: wiredTiger
  wiredTiger:
    engineConfig:
        cacheSizeGB: 2

As you can see I have 8GB of RAM and even now Mongo is on the verge of consuming most of it. By the time I posted this OOM had already intervened:

Mar  6 13:19:13 host kernel: [ 5638.313813] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/mongod.service,task=mongod,pid=13997,uid=116
Mar  6 13:19:13 host kernel: [ 5638.314025] Out of memory: Killed process 13997 (mongod) total-vm:8773352kB, anon-rss:6775648kB, file-rss:0kB, shmem-rss:0kB, UID:116 pgtables:14084kB oom_score_adj:0

I realize cacheSizeGB may not be the only factor to consider here, but what else should I be looking at?

@billy_noah not sure, if you are still stuck … but here a few things I can suggest

To address the issue of MongoDB consuming all available RAM, apart from cacheSizeGB, one can consider other factors such as the size of the database and the number of connections to the server. It is also worth checking if there are any poorly written queries that are not optimized and are causing the database to use a lot of memory. Another possible issue could be that the hardware resources are insufficient for the workload.
To further diagnose the issue, one can check the MongoDB logs for any warnings or errors related to memory usage. It is also recommended to monitor the memory usage of the server and the mongod process over time to understand how the consumption is changing. Additionally, it is worth considering provisioning a swap space to prevent the mongod process from being killed by the OOM killer.