- Administration >
- MongoDB Performance
MongoDB Performance¶
On this page
As you develop and operate applications with MongoDB, you may need to analyze the performance of the application and its database. When you encounter degraded performance, it is often a function of database access strategies, hardware availability, and the number of open database connections.
Some users may experience performance limitations as a result of inadequate or inappropriate indexing strategies, or as a consequence of poor schema design patterns. Locking Performance discusses how these can impact MongoDB’s internal locking.
Performance issues may indicate that the database is operating at capacity and that it is time to add additional capacity to the database. In particular, the application’s working set should fit in the available physical memory. See Memory and the MMAPv1 Storage Engine for more information on the working set.
In some cases performance issues may be temporary and related to abnormal traffic load. As discussed in Number of Connections, scaling can help relax excessive traffic.
Database Profiling can help you to understand what operations are causing degradation.
Locking Performance¶
MongoDB uses a locking system to ensure data set consistency. If certain operations are long-running or a queue forms, performance will degrade as requests and operations wait for the lock.
Lock-related slowdowns can be intermittent. To see if the lock has been
affecting your performance, refer to the locks
section and the globalLock section of the
serverStatus
output.
Dividing locks.timeAcquiringMicros
by
locks.acquireWaitCount
can give an approximate average wait time for a particular lock mode.
locks.deadlockCount
provide
the number of times the lock acquisitions encountered deadlocks.
If globalLock.currentQueue.total
is consistently high,
then there is a chance that a large number of requests are waiting for
a lock. This indicates a possible concurrency issue that may be affecting
performance.
If globalLock.totalTime
is
high relative to uptime
, the database has
existed in a lock state for a significant amount of time.
Long queries can result from ineffective use of indexes; non-optimal schema design; poor query structure; system architecture issues; or insufficient RAM resulting in page faults and disk reads.
Memory and the MMAPv1 Storage Engine¶
Memory Use¶
With the MMAPv1 storage engine, MongoDB uses
memory-mapped files to store data. Given a data set of sufficient size,
the mongod
process will allocate all available memory on the system
for its use.
While this is intentional and aids performance, the memory mapped files make it difficult to determine if the amount of RAM is sufficient for the data set.
The memory usage statuses metrics of the
serverStatus
output can provide insight into MongoDB’s
memory use.
The mem.resident
field provides the
amount of resident memory in use. If this exceeds the amount of system
memory and there is a significant amount of data on disk that isn’t in RAM,
you may have exceeded the capacity of your system.
You can inspect mem.mapped
to check the
amount of mapped memory that mongod
is using. If this value is
greater than the amount of system memory, some operations will require a
page faults to read data from disk.
Page Faults¶
With the MMAPv1 storage engine, page faults can occur as MongoDB reads from or writes data to parts of its data files that are not currently located in physical memory. In contrast, operating system page faults happen when physical memory is exhausted and pages of physical memory are swapped to disk.
MongoDB reports its triggered page faults as the total number of
page faults in one second. To check for page faults, see
the extra_info.page_faults
value
in the serverStatus
output.
Rapid increases in the MongoDB page fault counter may indicate that the server has too little physical memory. Page faults also can occur while accessing large data sets or scanning an entire collection.
A single page fault completes quickly and is not problematic. However, in aggregate, large volumes of page faults typically indicate that MongoDB is reading too much data from disk.
MongoDB can often “yield” read locks after a page fault, allowing other database
processes to read while mongod
loads the next page into memory.
Yielding the read lock following a page fault improves concurrency, and also
improves overall throughput in high volume systems.
Increasing the amount of RAM accessible to MongoDB may help reduce the
frequency of page faults. If this is not possible, you may want to consider
deploying a sharded cluster or adding shards
to your deployment to distribute load among mongod
instances.
See What are page faults? for more information.
Number of Connections¶
In some cases, the number of connections between the applications and the
database can overwhelm the ability of the server to handle requests. The
following fields in the serverStatus
document can provide insight:
connections
is a container for the following two fields:connections.current
the total number of current clients connected to the database instance.connections.available
the total number of unused connections available for new clients.
If there are numerous concurrent application requests, the database may have trouble keeping up with demand. If this is the case, then you will need to increase the capacity of your deployment.
For write-heavy applications, deploy sharding and add one or more
shards to a sharded cluster to distribute load among
mongod
instances.
Spikes in the number of connections can also be the result of application or driver errors. All of the officially supported MongoDB drivers implement connection pooling, which allows clients to use and reuse connections more efficiently. Extremely high numbers of connections, particularly without corresponding workload is often indicative of a driver or other configuration error.
Unless constrained by system-wide limits, MongoDB has no limit on
incoming connections. On Unix-based systems, you can modify system limits
using the ulimit
command, or by editing your system’s
/etc/sysctl
file. See UNIX ulimit Settings for more
information.
Database Profiling¶
The Database Profiler collects detailed information about operations run against a mongod instance. The profiler’s output can help to identify inefficient queries and operations.
You can enable and configure profiling for individual databases or for
all databases on a mongod
instance.
Profiler settings affect only a single mongod
instance and
will not propagate across a replica set or sharded
cluster.
See Database Profiler for information on enabling and configuring the profiler.
The following profiling levels are available:
Level | Description |
---|---|
0 |
The profiler is off and does not collect any data. This is the default profiler level. |
1 |
The profiler collects data for operations that take longer
than the value of slowms . |
2 |
The profiler collects data for all operations. |
Important
Profiling can impact performance and shares settings with the system log. Carefully consider any performance and security implications before configuring and enabling the profiler on a production deployment.
See Profiler Overhead for more information on potential performance degradation.
Note
When logLevel
is set to 0
, MongoDB records slow
operations to the diagnostic log at a rate determined by
slowOpSampleRate
. For MongoDB 3.6
deployments, starting in version 3.6.11, the secondaries of replica
sets log all oplog entry messages that take longer than the slow
operation threshold to apply regardless of the sample
rate.
At higher logLevel
settings, all operations appear in
the diagnostic log regardless of their latency with the following
exception: the logging of slow oplog entry messages by the
secondaries. The secondaries log only the slow oplog
entries; increasing the logLevel
does not log all
oplog entries.
Full Time Diagnostic Data Capture¶
To facilitate analysis of the MongoDB server behavior by MongoDB Inc.
engineers, mongod
and mongos
processes include a
Full Time Diagnostic Data Collection (FTDC) mechanism. FTDC data files
are compressed, are not human-readable, and inherit the same file access
permissions as the MongoDB data files. Only users with access to FTDC
data files can transmit the FTDC data. MongoDB Inc. engineers cannot
access FTDC data independent of system owners or operators. MongoDB
processes run with FTDC on by default. For more information on MongoDB
Support options, visit
Getting Started With MongoDB Support.
FTDC Privacy
FTDC data files are compressed and not human-readable. MongoDB Inc. engineers cannot access FTDC data without explicit permission and assistance from system owners or operators.
FTDC data never contains any of the following information:
- Samples of queries, query predicates, or query results
- Data sampled from any end-user collection or index
- System or MongoDB user credentials or security certificates
FTDC data contains certain host machine information such as
hostnames, operating system information, and the options or settings
used to start the mongod
or
mongos
. This information may be
considered protected or confidential by some organizations or
regulatory bodies, but is not typically considered to be Personally
Identifiable Information (PII). For clusters where these fields were
configured with protected, confidential, or PII data, please notify
MongoDB Inc. engineers before sending the FTDC data so appropriate
measures can be taken.
FTDC periodically collects statistics produced by the following commands:
serverStatus
replSetGetStatus
(mongod
only)collStats
for thelocal.oplog.rs
collection (mongod
only)connPoolStats
(mongos
only)
Depending on the host operating system, the diagnostic data may include one or more of the following statistics:
- CPU utilization
- Memory utilization
- Disk utilization related to performance. FTDC does not include data related to storage capacity.
- Network performance statistics. FTDC only captures metadata and does not capture or inspect any network packets.
FTDC collects statistics produced by the following commands on file rotation or startup:
mongod
processes store FTDC data files in a
diagnostic.data
directory under the instances
storage.dbPath
. All diagnostic data files are stored
under this directory. For example, given a dbPath
of /data/db
, the diagnostic data directory would be
/data/db/diagnostic.data
.
mongos
processes store FTDC data files in a
diagnostic directory relative to the systemLog.path
log
path setting. MongoDB truncates the logpath’s file extension and
concatenates diagnostic.data
to the remaining name. For example,
given a path
setting of
/var/log/mongodb/mongos.log
, the diagnostic data directory would be
/var/log/mongodb/mongos.diagnostic.data
.
FTDC runs with the following defaults:
- Data capture every 1 second
- 200MB maximum
diagnostic.data
folder size.
These defaults are designed to provide useful data to MongoDB Inc. engineers with minimal impact on performance or storage size. These values only require modifications if requested by MongoDB Inc. engineers for specific diagnostic purposes.
You can view the FTDC source code on the
MongoDB Github Repository.
The ftdc_system_stats_*.ccp
files specifically define any
system-specific diagnostic data captured.
To disable FTDC, start up the mongod
or
mongos
with the
diagnosticDataCollectionEnabled: false
option specified to the
setParameter
setting in your configuration file:
Disabling FTDC may increase the time or resources required when analyzing or debugging issues with support from MongoDB Engineers.