Hi All,
At this moment I am experiencing issues with a Mongo replicaset version 4.0.12 and slowly increasing memory usage only on the primary node. Over the course of 3-4 weeks memory slowly grows from 30% to 99% after which the server becomes unresponsive and at a certain point steps down as primary. Data size does not change as all collections are cleaned up periodically using a client side script or capped collection. A similar setup which is processing more data is not having this issue. Restarting the node solves the issue for the next couple of weeks. Is there anything I should check or change?
Some more detailed information:
replicas have 2GiB memory and around 50GiB of data. Indexes are around 30MiB. No slow queries reported by Mongo. 60-70 connections are open at the same time. TcMalloc looks like this before crashing. wiredtiger cache only uses 500MiB.
MALLOC: 2091358456 ( 1994.5 MiB) Bytes in use by application\nMALLOC: + 96292864 ( 91.8 MiB) Bytes in page heap freelist\nMALLOC: + 43007552 ( 41.0 MiB) Bytes in central cache freelist\nMALLOC: + 4904832 ( 4.7 MiB) Bytes in transfer cache freelist\nMALLOC: + 21930312 ( 20.9 MiB) Bytes in thread cache freelists\nMALLOC: + 19816704 ( 18.9 MiB) Bytes in malloc metadata\nMALLOC: ------------\nMALLOC: = 2277310720 ( 2171.8 MiB) Actual memory used (physical + swap)\nMALLOC: + 345247744 ( 329.3 MiB) Bytes released to OS (aka unmapped)\nMALLOC: ------------\nMALLOC: = 2622558464 ( 2501.1 MiB) Virtual address space used\nMALLOC:\nMALLOC: 273100 Spans in use\nMALLOC: 86 Thread heaps in use\nMALLOC: 4096 Tcmalloc page size\n------------------------------------------------\nCall ReleaseFreeMemory() to release freelist memory to the OS (via madvise()).\nBytes released to the OS take up virtual address space but no physical memory.
A possible related issue is https://jira.mongodb.org/browse/SERVER-43632