- Sharding >
- Sharded Cluster Administration
Sharded Cluster Administration¶
On this page
Sharding occurs within a sharded cluster. A sharded cluster consists of the following components:
- Shards. Each shard is a separate
mongod
instance or replica set that holds a portion of the database collections. - Config servers. Each config server is
a
mongod
instance that holds metadata about the cluster. The metadata maps chunks to shards. - mongos instances. The
mongos
instances route the reads and writes to the shards.
See also
- For specific configurations, see Sharded Cluster Architectures.
- To set up sharded clusters, see Deploy a Sharded Cluster.
Shards¶
A shard is a container that holds a subset of a collection’s data. Each
shard is either a single mongod
instance or a replica
set. In production, all shards should be replica sets.
Applications do not access the shards directly. Instead, the mongos instances routes reads and writes from applications to the shards.
Config Servers¶
Config servers maintain the shard metadata in a config database. The
config database stores the relationship between chunks and where they reside within a sharded cluster. Without
a config database, the mongos
instances would be unable to
route queries or write operations within the cluster.
Config servers do not run as replica sets. Instead, a cluster operates with a group of three config servers that use a two-phase commit process that ensures immediate consistency and reliability.
For testing purposes you may deploy a cluster with a single config server, but this is not recommended for production.
Warning
If your cluster has a single config server, this
mongod
is a single point of failure. If the instance is
inaccessible the cluster is not accessible. If you cannot recover
the data on a config server, the cluster will be inoperable.
Always use three config servers for production deployments.
The actual load on configuration servers is small because each
mongos
instance maintains a cached copy of the configuration
database. MongoDB only writes data to the config server to:
- create splits in existing chunks, which happens as data in existing chunks exceeds the maximum chunk size.
- migrate a chunk between shards.
Additionally, all config servers must be available on initial setup
of a sharded cluster, each mongos
instance must be able
to write to the config.version
collection.
If one or two configuration instances become unavailable, the cluster’s metadata becomes read only. It is still possible to read and write data from the shards, but no chunk migrations or splits will occur until all three servers are accessible. At the same time, config server data is only read in the following situations:
- A new
mongos
starts for the first time, or an existingmongos
restarts. - After a chunk migration, the
mongos
instances update themselves with the new cluster metadata.
If all three config servers are inaccessible, you can continue to use
the cluster as long as you don’t restart the mongos
instances until after config servers are accessible again. If you
restart the mongos
instances and there are no accessible
config servers, the mongos
would be unable to direct
queries or write operations to the cluster.
Because the configuration data is small relative to the amount of data stored in a cluster, the amount of activity is relatively low, and 100% up time is not required for a functioning sharded cluster. As a result, backing up the config servers is not difficult. Backups of config servers are critical as clusters become totally inoperable when you lose all configuration instances and data. Precautions to ensure that the config servers remain available and intact are critical.
Note
Configuration servers store metadata for a single sharded cluster. You must have a separate configuration server or servers for each cluster you administer.
Sharded Cluster Operations and mongos
Instances¶
The mongos
program provides a single unified interface to a sharded
cluster for applications using MongoDB. Except for the selection of a
shard key, application developers and administrators need not
consider any of the internal details of sharding.
mongos
caches data from the config server, and uses this to route operations from
applications and clients to the mongod
instances.
mongos
have no persistent state and consume
minimal system resources.
The most common practice is to run mongos
instances on the
same systems as your application servers, but you can maintain
mongos
instances on the shards or on other dedicated
resources.
Note
Changed in version 2.1.
Some aggregation operations using the aggregate
command (i.e. db.collection.aggregate()
,) will cause
mongos
instances to require more CPU resources than in
previous versions. This modified performance profile may dictate
alternate architecture decisions if you use the aggregation
framework extensively in a sharded environment.
Automatic Operation and Query Routing with mongos
¶
mongos
uses information from config servers to route operations to the cluster as
efficiently as possible. In general, operations in a sharded
environment are either:
- Targeted at a single shard or a limited group of shards based on the shard key.
- Broadcast to all shards in the cluster that hold documents in a collection.
When possible you should design your operations to be as targeted as possible. Operations have the following targeting characteristics:
Query operations broadcast to all shards [1] unless the
mongos
can determine which shard or shard stores this data.For queries that include the shard key,
mongos
can target the query at a specific shard or set of shards, if the portion of the shard key included in the query is a prefix of the shard key. For example, if the shard key is:The
mongos
program can route queries that include the full shard key or either of the following shard key prefixes at a specific shard or set of shards:Depending on the distribution of data in the cluster and the selectivity of the query,
mongos
may still have to contact multiple shards [2] to fulfill these queries.All
insert()
operations target to one shard.All single
update()
operations target to one shard. This includes upsert operations.The
mongos
broadcasts multi-update operations to every shard.The
mongos
broadcastsremove()
operations to every shard unless the operation specifies the shard key in full.
While some operations must broadcast to all shards, you can improve performance by using as many targeted operations as possible by ensuring that your operations include the shard key.
[1] | If a shard does not store chunks from a given collection, queries for documents in that collection are not broadcast to that shard. |
[2] | mongos will route some queries, even
some that include the shard key, to all shards, if needed. |
Sharded Query Response Process¶
To route a query to a cluster,
mongos
uses the following process:
Determine the list of shards that must receive the query.
In some cases, when the shard key or a prefix of the shard key is a part of the query, the
mongos
can route the query to a subset of the shards. Otherwise, themongos
must direct the query to all shards that hold documents for that collection.Example
Given the following shard key:
Depending on the distribution of chunks in the cluster, the
mongos
may be able to target the query at a subset of shards, if the query contains the following fields:Establish a cursor on all targeted shards.
When the first batch of results returns from the cursors:
For query with sorted results (i.e. using
cursor.sort()
) themongos
instance performs a merge sort of all queries.For a query with unsorted results, the
mongos
instance returns a result cursor that “round robins” results from all cursors on the shards.Changed in version 2.0.5: Before 2.0.5, the
mongos
exhausted each cursor, one by one.
Sharded Cluster Security Considerations¶
MongoDB controls access to sharded clusters
with key files that store authentication credentials. The components
of sharded clusters use the secret stored in the key files when
authenticating to each other. Create key files and then point your
mongos
and mongod
instances to the files, as
described later in this section.
Beyond the auth
mechanisms described in this section,
always run your sharded clusters in trusted networking environments
that limit access to the cluster with network rules. Your networking
environments should enforce restrictions that ensure only known
traffic reaches your mongos
and mongod
instances.
This section describes authentication specific to sharded clusters. For information on authentication across MongoDB, see Authentication.
Access Control Privileges in Sharded Clusters¶
In sharded clusters, MongoDB provides separate administrative privileges for the sharded cluster and for each shard. Beyond these administration privileges, privileges for sharded cluster deployments are functionally the same as any other MongoDB deployment. See, Authentication for more information.
For sharded clusters, MongoDB provides these separate administrative privileges:
Administrative privileges for the sharded cluster. These privileges provide read-and-write access to the config servers’ ‘admin. These users can run all administrative commands. Administrative privileges also give the user read-and-write access to all the cluster’s databases.
The credentials for administrative privileges on the cluster reside on the config servers. To receive admin access to the cluster, you must authenticate a session while connected to a
mongos
instance using the admin database.Administrative privileges for the
mongod
instance, or replica set, that provides each individual shard. Each shard has its own admin database that stores administrative credentials and access for that shard only. These credentials are completely distinct from the cluster-wide administrative credentials.As with all
mongod
instances, MongoDB provides two types of administrative privileges for a shard:- Normal administrative privileges, which provide read-and-write access to the admin database and access to all administrative commands, and which provide read-and-write access to all other databases on that shard.
- Read-only administrative privileges, which provide read-only access to the admin database and to all other databases on that shard.
Also, as with all mongod
instances, a MongoDB sharded cluster
provides the following non-administrative user privileges:
- Normal privileges, which provide read-and-write access to a specific database. Users with normal privilege can add users to the database.
- Read-only privileges, which provide read-only access to a specific database.
For more information on privileges, see Authentication.
Enable Authentication in a Sharded Cluster¶
New in version 2.0: Support for authentication with sharded clusters.
To control access to a sharded cluster, create key files and then set
the keyFile
option on all components of the sharded
cluster, including all mongos
instances, all config server
mongod
instances, and all shard mongod
instances. The content of the key file is arbitrary but must be the
same on all cluster members.
To enable authentication, do the following:
Generate a key file to store authentication information, as described in the Generate a Key File section.
On each component in the sharded cluster, enable authentication by doing one of the following:
Add the first administrative user and then add subsequent users. See Add Users.
Access a Sharded Cluster with Authentication¶
To access a sharded cluster as an authenticated admin user, see Administrative Access in MongoDB.
To access a sharded cluster as an authenticated, non-admin user, see either of the following:
To terminate an authenticated session, see the logout
command.