What is the relationship between wiredtiger cache and operation system cache？

Ops and Admin
#1

Dear,

what is the relationship between wiredtiger cache and operation system cache？

wiredtiger cache: stores uncompressed data？
operation system cache：stores compress data?
then could you explain how mongo use wiredtiger cache and operation system cache of the read/write operation.

read:
1> if data is in wiredtiger cache, then read from this cache
2> if data is not in wiredtiger cache but in operation system cache: then read from operation system cache into wiredtiger cache and uncompress it? and finally return to client?
3> if data is in disk only, then read from disk to operation system cache first ,then read from operation system cache to wiredtiger cache and uncompress ?

write:
write to wiredcache first then to operation system cache and finally to disk？？？

1 Like
#2

Hi @Yi_deng

In general your assessments are correct. WiredTiger cache stores the uncompressed documents, while OS-managed filesystem cache stored the compressed documents. Both caches are used during normal operations. However please note that the OS filesystem cache is managed by the OS, and WiredTiger has no control over it.

In terms of writes, WiredTiger writes to its on-disk journal, before syncing the modifications to the data files on checkpoints that occur every 60 seconds. Please see the Journal page linked above for details on when the writes are flushed to disk during the journaling process.

Best regards
Kevin

1 Like
#3

Dear kevin,
thanks.
so mongodb has two level buffers, one is wiredtiger ,another is operation system.
some operations in mongodb will go through the two level buffers.
if my active data + indexes are 10GB, do i need 10GB wiredtiger cache + 10GB operation system memory(maybe less than 10GB because mongo`s data compession stored in operation system buffer) = at least 20GB memory ?

mongodb uses 50%*(physical memory -1 GB) for wiredtiger cache by default, is this a consideration for leaving another 50% physical memory for operation system cache?

and if I do not reserve enough memory for operation system, for example, i have 30GB active data+indexes,after i config wiredtigercache to 30GB, the operation system has only 2GB free memory , then what is the impact to performance?

#4

so mongodb has two level buffers, one is wiredtiger ,another is operation system.

The OS filesystem cache is not a MongoDB feature, but in loose terms, yes.

if my active data + indexes are 10GB, do i need 10GB wiredtiger cache + 10GB operation system memory(maybe less than 10GB because mongo`s data compession stored in operation system buffer) = at least 20GB memory ?

I assume the 10GB is your working set? Yes that is correct. So if you worked out that the working set is 10GB, it’s ideal to set the WiredTiger cache to be at least that number. The other ~10GB (more on this later) in the fileystem cache contains compressed data, so what it contains depends highly on how compressible your data is.

mongodb uses 50%*(physical memory -1 GB) for wiredtiger cache by default, is this a consideration for leaving another 50% physical memory for operation system cache?

About 50% of RAM for WiredTiger cache is mostly sufficient to cover the most typical use cases, but of course some workload may require more, and some probably can do with less. The 50% number was deemed to be a good compromise in most cases by balancing the needs of WiredTiger vs. the needs of the OS. Since MongoDB has no control over how the OS manages the filesystem cache, it’s entirely possible that the cache don’t fill the other 50% entirely. This would probably be different from OS to OS.

Best regards
Kevin