- Replication >
- Deploy a Geographically Distributed Replica Set
Deploy a Geographically Distributed Replica Set¶
On this page
This tutorial outlines the process for deploying a replica set with members in multiple locations. The tutorial addresses three-member sets, four-member sets, and sets with more than four members.
For appropriate background, see Replica Set Fundamental Concepts and Replica Set Architectures and Deployment Patterns. For related tutorials, see Deploy a Replica Set and Add Members to a Replica Set.
Overview¶
While replica sets provide basic protection against single-instance failure, when all of the members of a replica set reside in a single facility, the replica set is still susceptible to some classes of errors in that facility including power outages, networking distortions, and natural disasters. To protect against these classes of failures, deploy a replica set with one or more members in a geographically distinct facility or data center.
Requirements¶
If possible, use an odd number of data centers, and choose a distribution of members that maximizes the likelihood that even with a loss of a data center, the remaining replica set members can form a majority or at minimum, provide a copy of your data.
If you deploy a replica set with an uneven number of members, add an arbiter.
For all configurations in this tutorial, deploy each replica set member on a separate system. Although you may deploy more than one replica set member on a single system, doing so reduces the redundancy and capacity of the replica set. Such deployments are typically for testing purposes and beyond the scope of this tutorial.
Procedures¶
Deploy a Distributed Three-Member Replica Set¶
A geographically distributed three-member deployment has the following features:
Each member of the replica set resides on its own machine, and the MongoDB processes all bind to port
27017
, which is the standard MongoDB port.Each member of the replica set must be accessible by way of resolvable DNS or hostnames in the following scheme:
mongodb0.example.net
mongodb1.example.net
mongodb2.example.net
Configure DNS names appropriately, or set up your systems’
/etc/hosts
file to reflect this configuration.Ensure that network traffic can pass between all members in the network securely and efficiently. Consider the following:
Establish a virtual private network between the systems in Site A and Site B to encrypt all traffic between the sites and remains private. Ensure that your network topology routes all traffic between members within a single site over the local area network.
Configure authentication using
auth
andkeyFile
, so that only servers and process with authentication can connect to the replica set.Configure networking and firewall rules so that only traffic (incoming and outgoing packets) on the default MongoDB port (e.g.
27017
) from within your deployment.See also
For more information on security and firewalls, see Security Considerations for Replica Sets.
Specify run-time configuration on each system in a configuration file stored in
/etc/mongodb.conf
or in a related location. Do not specify run-time configuration through command line options.For each MongoDB instance, use the following configuration, with values set appropriate to your systems:
Modify
bind_ip
to reflect a secure interface on your system that is able to access all other members of the set and that is accessible to all other members of the replica set. The DNS or host names need to point and resolve to this IP address. Configure network rules or a virtual private network (i.e. “VPN”) to permit this access.Note
The portion of the
replSet
following the/
provides a “seed list” of known members of the replica set.mongod
uses this list to fetch configuration changes following restarts. It is acceptable to omit this section entirely, and have thereplSet
option resemble:For more documentation on the above run time configurations, as well as additional configuration options, see Configuration File Options.
To deploy a geographically distributed three-member set:
On each system start the
mongod
process by issuing a command similar to following:Note
In production deployments you likely want to use and configure a control script to manage this process based on this command. Control scripts are beyond the scope of this document.
Open a
mongo
shell connected to one of themongod
instances:Use the
rs.initiate()
method on one member to initiate a replica set consisting of the current member and using the default configuration:Display the current replica configuration:
Add the remaining members to the replica set by issuing a sequence of commands similar to the following. The example commands assume the current primary is
mongodb0.example.net
:Optional. Configure the member eligibility for becoming primary. In some cases, you may prefer that the members in one data center be elected primary before the members in the other data centers.
Issue the following command to determine the
_id
value formongodb2.example.net
:In the
members
array, find the array index of the member to configure. The example in the next step assumes this value is2
.In the
mongo
shell connected to the replica set’s primary, issue a command sequence similar to the following:Note
In some situations, the
rs.reconfig()
shell method can force the current primary to step down and causes an election. When the primary steps down, all clients will disconnect. This is the intended behavior. While, this typically takes 10-20 seconds, attempt to make these changes during scheduled maintenance periods.
After these commands return you have a geographically distributed three-member replica set.
To check the status of your replica set, issue
rs.status()
.
See also
The documentation of the following shell functions for more information:
Deploy a Distributed Four-Member Replica Set¶
A geographically distributed four-member deployment has the following features:
Each member of the replica set, except for the arbiter (see below), resides on its own machine, and the MongoDB processes all bind to port
27017
, which is the standard MongoDB port.Each member of the replica set must be accessible by way of resolvable DNS or hostnames in the following scheme:
mongodb0.example.net
mongodb1.example.net
mongodb2.example.net
mongodb3.example.net
Configure DNS names appropriately, or set up your systems’
/etc/host
file to reflect this configuration.One host (e.g.
mongodb3.example.net
) will be an arbiter and can run on a system that is also used for an application server or some other shared purpose.There are three possible architectures for this replica set:
- Two members in Site A, two secondary-only members in Site B, and an arbiter in Site A.
- Three members in Site A and one secondary-only member in Site B.
- Two members in Site A, one secondary-only member in Site B, one secondary-only member in Site C, and an arbiter in site A.
In most cases the first architecture is preferable because it is the least complex.
Ensure that network traffic can pass between all members in the network securely and efficiently. Consider the following:
Establish a virtual private network between the systems in Site A and Site B (and Site C if it exists) to encrypt all traffic between the sites and remains private. Ensure that your network topology routes all traffic between members within a single site over the local area network.
Configure authentication using
auth
andkeyFile
, so that only servers and process with authentication can connect to the replica set.Configure networking and firewall rules so that only traffic (incoming and outgoing packets) on the default MongoDB port (e.g.
27017
) from within your deployment.See also
For more information on security and firewalls, see Security Considerations for Replica Sets.
Specify run-time configuration on each system in a configuration file stored in
/etc/mongodb.conf
or in a related location. Do not specify run-time configuration through command line options.For each MongoDB instance, use the following configuration, with values set appropriate to your systems:
Modify
bind_ip
to reflect a secure interface on your system that is able to access all other members of the set and that is accessible to all other members of the replica set. The DNS or host names need to point and resolve to this IP address. Configure network rules or a virtual private network (i.e. “VPN”) to permit this access.Note
The portion of the
replSet
following the/
provides a “seed list” of known members of the replica set.mongod
uses this list to fetch configuration changes following restarts. It is acceptable to omit this section entirely, and have thereplSet
option resemble:For more documentation on the above run time configurations, as well as additional configuration options, see doc:/reference/configuration-options.
To deploy a geographically distributed four-member set:
On each system start the
mongod
process by issuing a command similar to following:Note
In production deployments you likely want to use and configure a control script to manage this process based on this command. Control scripts are beyond the scope of this document.
Open a
mongo
shell connected to this host:Use
rs.initiate()
to initiate a replica set consisting of the current member and using the default configuration:Display the current replica configuration:
Add the remaining members to the replica set by issuing a sequence of commands similar to the following. The example commands assume the current primary is
mongodb0.example.net
:In the same shell session, issue the following command to add the arbiter (e.g.
mongodb4.example.net
):Optional. Configure the member eligibility for becoming primary. In some cases, you may prefer that the members in one data center be elected primary before the members in the other data centers.
Issue the following command to determine the
_id
value for the member:In the
members
array, find the array index of the member to configure. The example in the next step assumes this value is2
.In the
mongo
shell connected to the replica set’s primary, issue a command sequence similar to the following:Note
In some situations, the
rs.reconfig()
shell method can force the current primary to step down and causes an election. When the primary steps down, all clients will disconnect. This is the intended behavior. While, this typically takes 10-20 seconds, attempt to make these changes during scheduled maintenance periods.
After these commands return you have a geographically distributed four-member replica set.
To check the status of your replica set, issue
rs.status()
.
See also
The documentation of the following shell functions for more information:
Deploy a Distributed Set with More than Four Members¶
The above procedures detail the steps necessary for deploying a geographically redundant replica set. Larger replica set deployments generally follow the same steps, but have additional considerations:
- Never deploy more than seven voting members.
- If you have an even number of members, similar to the procedure for a four-member set, deploy an arbiter.