Docs Menu
Docs Home
/
MongoDB Manual
/ / /

TCMalloc Performance Optimization for a Self-Managed Deployment

On this page

  • Platform Support
  • Enable Transparent Hugepages (THP)
  • Enable Per-CPU Caches
  • Learn More

Changed in version 8.0.

Starting in MongoDB 8.0, MongoDB uses an upgraded version of TCMalloc that uses per-CPU caches, instead of per-thread caches, to reduce memory fragmentation and make your database more resilient to high-stress workloads.

To use the new TCMalloc version:

  • Use a supported operating system.

  • Enable THP.

  • Enable per-CPU caches.

Operating systems that support MongoDB 8.0 also support the updated TCMalloc, except for the following:

  • RHEL 8 / CentOS 8 / Oracle 8 on the PPC64LE and s390x architectures

  • RHEL 9 / CentOS 9 / Oracle 9 on the PPC64LE architecture

These operating systems use the legacy TCMalloc version. If you use these operating systems, disable THP.

Transparent Hugepages (THP) is a Linux memory management system that reduces the overhead of Translation Lookaside Buffer (TLB) lookups. THP achieves this by combining small pages and making them appear as larger memory pages to the application.

In MongoDB 8.0 and later, ensure that THP is enabled before mongod starts by creating a service file for your platform's initialization system. If you are using MongoDB 7.0 or earlier, disable THP.

Note

You can enable THP at the system level and disable it on the process level. If you have multiple MongoDB processes on a single machine, ensure that processes on version 8.0 enable THP, while processes on 7.0 or earlier disable THP.

Additionally, for RHEL and CentOS systems that use ktune and tuned performance profiles, you must also create a custom tuned profile.

To create a service file that enables THP, use the built-in initialization system for your platform. Recent versions of Linux typically use systemd, which uses the systemctl command. Older versions of Linux tend to use System V init, which uses the service command. For more information, see the documentation for your operating system.

Use the initialization system for your platform:

1

Create the following file and save it at /etc/systemd/system/enable-transparent-huge-pages.service:

[Unit]
Description=Enable Transparent Hugepages (THP)
DefaultDependencies=no
After=sysinit.target local-fs.target
Before=mongod.service
[Service]
Type=oneshot
ExecStart=/bin/sh -c 'echo always | tee /sys/kernel/mm/transparent_hugepage/enabled > /dev/null && echo defer+madvise | tee /sys/kernel/mm/transparent_hugepage/defrag > /dev/null && echo 0 | tee /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none > /dev/null && echo 1 | tee /proc/sys/vm/overcommit_memory > /dev/null'
[Install]
WantedBy=basic.target

Note

Some versions of Red Hat Enterprise Linux, and potentially other Red Hat-based derivatives, use a different path for the THP enabled file:

/sys/kernel/mm/redhat_transparent_hugepage/enabled

Verify which path is in use on your system and update the enable-transparent-huge-pages.service file accordingly.

2

To reload the systemd unit files and make enable-transparent-huge-pages.service available for use, run the following command :

sudo systemctl daemon-reload
3

Run:

sudo systemctl start enable-transparent-huge-pages

To verify that the relevant THP settings have changed, run the following command:

cat /sys/kernel/mm/transparent_hugepage/enabled && cat /sys/kernel/mm/transparent_hugepage/defrag && cat /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none && cat /proc/sys/vm/overcommit_memory

On Red Hat Enterprise Linux and potentially other Red Hat-based derivatives, you may instead need to use the following:

cat /sys/kernel/mm/redhat_transparent_hugepage/enabled && cat /sys/kernel/mm/redhat_transparent_hugepage/defrag && cat /sys/kernel/mm/redhat_transparent_hugepage/khugepaged/max_ptes_none && cat /proc/sys/vm/overcommit_memory

The output should resemble the following:

always
defer+madvise
0
1
4

To ensure that this setting is applied each time the operating system starts, run the following command:

sudo systemctl enable enable-transparent-huge-pages
5

If you use tuned or ktune proffiles on RHEL/ CentOS, you must also create a custom tuned profile.

1

Create the following file and save it at /etc/init.d/enable-transparent-hugepages:

#!/bin/bash
### BEGIN INIT INFO
# Provides: enable-transparent-hugepages
# Required-Start: $local_fs
# Required-Stop:
# X-Start-Before: mongod mongodb-mms-automation-agent
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Enable Linux Transparent Hugepages
# Description: Enable Linux Transparent Hugepages, to improve
# database performance.
### END INIT INFO
case $1 in
start)
if [ -d /sys/kernel/mm/transparent_hugepage ]; then
thp_path=/sys/kernel/mm/transparent_hugepage
elif [ -d /sys/kernel/mm/redhat_transparent_hugepage ]; then
thp_path=/sys/kernel/mm/redhat_transparent_hugepage
else
return 0
fi
echo 'always' | tee ${thp_path}/enabled > /dev/null && echo defer+madvise | tee ${thp_path}/defrag > /dev/null && echo 0 | tee ${thp_path}/khugepaged/max_ptes_none > /dev/null && echo 1 | tee /proc/sys/vm/overcommit_memory > /dev/null'
unset thp_path
;;
esac
2

Run:

sudo chmod 755 /etc/init.d/enable-transparent-hugepages
3

Run:

sudo /etc/init.d/enable-transparent-hugepages start

To verify that the relevant THP settings have changed, run the following command:

cat /sys/kernel/mm/transparent_hugepage/enabled && cat /sys/kernel/mm/transparent_hugepage/defrag && cat /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none && cat /proc/sys/vm/overcommit_memory

On Red Hat Enterprise Linux and potentially other Red Hat-based derivatives, you may instead need to use the following:

cat /sys/kernel/mm/redhat_transparent_hugepage/enabled

The output should resemble the following:

always
defer+madvise
0
1
4

To ensure that this setting is applied each time the operating sytem starts, run the following command for your Linux distribution:

Distribution
Command
Ubuntu and Debian
sudo update-rc.d enable-transparent-hugepages defaults
SUSE
sudo insserv /etc/init.d/enable-transparent-hugepages
Red Hat, CentOS, Amazon Linux, and derivatives
sudo chkconfig --add enable-transparent-hugepages
5

If you are using tuned or ktune profiles on RHEL/ CentOS, you must also create a custom tuned profile.

Important

If you use tuned or ktune, perform the steps in this section after creating the service file.

tuned and ktune are kernel tuning utilities that can affect the Transparent Hugepages setting on your system. If you are using tuned or ktune on your RHEL or CentOS system while running mongod, you must create a custom tuned profile to ensure that THP stays enabled.

1

Create a new directory to store the custom tuned profile. The following example inherits from the existing virtual-guest profile, and uses virtual-guest-thp as the new profile:

sudo mkdir /etc/tuned/virtual-guest-thp
2

Create and edit /etc/tuned/virtual-guest-thp/tuned.conf. Add the following text:

[main]
include=virtual-guest
[vm]
transparent_hugepages=always

This example inherits from the existing virtual-guest profile. Select the appropriate profile for your system.

3

Run:

sudo tuned-adm profile virtual-guest-thp
1

To create a new profile from an existing profile, copy the relevant directory. The following example uses the virtual-guest profile as the base, and uses virtual-guest-thp as the new profile:

sudo cp -r /etc/tune-profiles/virtual-guest /etc/tune-profiles/virtual-guest-thp
2

Edit /etc/tune-profiles/virtual-guest-thp/ktune.sh and change the set_transparent_hugepages setting as follows:

set_transparent_hugepages always
3

Enable the new profile:

sudo tuned-adm profile virtual-guest-thp

To verify that TCMalloc is running with per-CPU caches, ensure that:

If per-CPU caches aren't enabled, ensure that:

  • You disable glibc rseq.

  • You're using Linux kernel version 4.18 or later.

The new TCMalloc requires Restartable Sequences (rseq) to implement per-CPU caches. If another application, such as the glibc library, registers an rseq structure before TCMalloc, TCMalloc can't use rseq. Without rseq, TCMalloc uses per-thread caches, which are used by the legacy TCMalloc version.

To ensure that TCMalloc can use rseq to enable per-CPU caches, you can disable glibc’s registration of a rseq structure. To disable glibc rseq, set the following environment variable before you start mongod:

GLIBC_TUNABLES=glibc.pthread.rseq=0
export GLIBC_TUNABLES

If you disabled glibc rseq and per-CPU caches are still not enabled, ensure that you're using Linux kernel version 4.18 or later. To check your kernel version, run the following command:

uname -r

Back

Performance