- Administer Ops Manager >
- Manage Snapshot Storage >
- Manage S3 Blockstore Snapshot Storage
Manage S3 Blockstore Snapshot Storage¶
On this page
Ops Manager can back up MongoDB databases as snapshots to one or more of the following storage options:
- Another MongoDB database, called a Blockstore,
- As files stored on a local or network-attached file system, and/or
- An AWS S3 bucket.
This tutorial covers backing up your MongoDB databases as snapshots stored in S3 buckets. The metadata for S3 snapshot stores is stored in a MongoDB database.
Note
You may have issues that require you to use more than one snapshot store like needing more capacity, localizing data, or meeting privacy regulations. To learn how to assign snapshot stores to different data centers, see Assign Snapshot Stores to Specific Data Centers.
Prerequisites¶
Metadata Storage Prerequisites¶
- Deploy the dedicated MongoDB instance(s) to serve the S3 snapshot store metadata and Oplog Store. Serve these instances on the same hosts as the Ops Manager server, the backing databases, or snapshot stores. Attach storage volume(s) with enough capacity to store the databases these instances manage.
- Ensure the host serving the Ops Manager Backup Daemon service has enough capacity to store the head database.
- Secure the instance that stores your S3 snapshot store metadata database using authentication and SSL. S3 snapshot store metadata databases support all authentication mechanisms.
S3 Storage Prerequisites¶
Make sure you have an IAM user on AWS.
Create your own AWS access keys for your IAM user. This allows you to create S3 buckets and store snapshot files in them. MongoDB does not create or issue AWS access keys.
Create your own S3 bucket to store your S3 snapshot store snapshots.
Note
The IAM user for which you created the AWS access keys must own the AWS S3 Bucket.
Add an S3 Blockstore¶
Click Create New S3 Blockstore.¶
Provide the S3 blockstore details.¶
Field | Contents | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Name | Type the name for the S3 blockstore. | |||||||||||||||
S3 Bucket Name | Type the name of the S3 bucket where you want to host the the S3 blockstore. Important You cannot move an S3 blockstore to another S3 bucket after the S3 blockstore has been created. If you need to use a different S3 bucket to host your S3 blockstore, you need to create a new S3 blockstore in that different S3 bucket. |
|||||||||||||||
AWS Access Key | Type your AWS Access Key ID. | |||||||||||||||
AWS Secret Key | Type your AWS Secret Access Key. | |||||||||||||||
Datastore Type | Select Standalone, Replica Set or Sharded Cluster. This MongoDB database stores the metadata for the blockstore. | |||||||||||||||
MongoDB Host List (replica set and sharded cluster only) | Type a comma-separated list of mongod instances
(for a Replica Set) or mongos instances (for a
Sharded Cluster) in the Example
|
|||||||||||||||
MongoDB Hostname (Standalone only) | Type the hostname of the S3 blockstore metadata database. | |||||||||||||||
MongoDB Port (Standalone only) | Type the port number of the S3 blockstore metadata database. | |||||||||||||||
Username |
|
|||||||||||||||
Password (optional) |
|
|||||||||||||||
Connection Options (optional) | Type any additional configuration file options for the MongoDB instance. This field supports unescaped values only. For proper syntax, see Connection String URI Format in the MongoDB manual. |
|||||||||||||||
Encrypted Credentials (optional) | Select if the credentials for the database were encrypted using the credentialstool. The credentials include the Username, Password, AWS Access Key ID and AWS Secret Key. | |||||||||||||||
Use TLS/SSL (optional) | Select if the S3 blockstore metadata database only accepts connection encrypted using TLS/SSL. To enable TLS on the S3 blockstore metadata database, see Configure mongod and mongos for TLS/SSL. |
|||||||||||||||
New Assignment Enabled (optional) | Select if you want to enable this S3 blockstore after creating it. This is selected by default so the S3 blockstore can be assigned backup jobs. If you clear this checkbox, the S3 blockstore is created but you cannot assign backups to this S3 blockstore. |
Click Create.¶
Edit an Existing S3 Blockstore¶
Once created, S3 snapshot stores are listed directly on the Snapshot Storage page in a table. Each row contains the settings for each S3 blockstore.
Navigate to the Snapshot Storage page.¶
- Click the Admin link.
- Click the Backup tab.
- (Optional) If you have not previously set the head directory, set it in the Head Directory box.
- Click the Snapshot Storage page.
Go to the row for the blockstore you want to edit.¶
In the MongoDB Connection column, update any values that need to be changed in the following fields:¶
Field | Contents | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Name | Type the name for the S3 blockstore. | |||||||||||||||
AWS Access Key | Type your AWS Access Key ID. | |||||||||||||||
AWS Secret Key | Type your AWS Secret Access Key. Note The existing Secret Access Key is not displayed. |
|||||||||||||||
<hostname>:<port> | Type in one or more hosts that comprise the S3 blockstore metadata
database in the Important If these hosts are changed, the blockstore they host must have the same data as the original blockstore. Changing the host to a new blockstore results in data loss.
|
|||||||||||||||
MongoDB Auth Username |
|
|||||||||||||||
MongoDB Auth Password |
Note The existing MongoDB Auth Password is not displayed. |
|||||||||||||||
Encrypted Credentials | Select if the credentials for the database were encrypted using the credentialstool. The credentials include the Username, Password, AWS Access Key ID and AWS Secret Key. | |||||||||||||||
Use TLS/SSL | Select if the blockstore database only accepts connection encrypted using TLS. To enable TLS on the S3 blockstore metadata database, see Configure mongod and mongos for TLS/SSL. |
|||||||||||||||
Connection Options | Type any additional configuration file options for the MongoDB instance. This field supports unescaped values only. For proper syntax, see Connection String URI Format in the MongoDB manual. |
|||||||||||||||
Assignment Labels (Optional) | Type a comma-separated list of labels to assign the S3 blockstores to specific groups. | |||||||||||||||
Load Factor (Optional) | Type any positive integer that expresses how much backup work you want this snapshot store to perform compared to another snapshot store. Important If you have only one snapshot store, skip this setting. Backup work includes running backups, restoring snapshots or grooming blockstores. The term of backup work ratio assigned to a single snapshot store is called its Load Factor. By default, Ops Manager assigns each snapshot store a
Load Factor of As a snapshot store’s Load Factor increases, it
performs more backup work compared to another snapshot store.
If the Load Factor of snapshot store Example How to estimate Load Factor Consider a five-shard sharded cluster with the following backup storage configuration:
In this example, Set the Load Factors of Snapshot stores with greater compute or storage performance should be given a greater Load Factor:
Load Factor can be set to |
|||||||||||||||
Write Concern | Select your preferred Write Concern:
|
Select the checkbox in the Assignment Enabled column.¶
Select if you want to enable this S3 blockstore after creating it. This is selected by default so the S3 blockstore can be assigned backup jobs. If you clear this checkbox, the S3 blockstore is created but you cannot assign backups to this S3 blockstore.
Click Save.¶
If you change any connection string values, restart all the Ops Manager instances including those running Backup Daemons.¶
Warning
Modifying the connection string values for an existing blockstore requires all Ops Manager components, including those only running the Backup Daemon, to be restarted to apply those changes. Connection parameters include:
- <hostname>:<port>,
- MongoDB Auth Username,
- MongoDB Auth Password,
- Encrypted Credentials,
- Use TLS/SSL, and
- Connection Options.
If you change to another blockstore host, the data on the existing blockstore is not copied automatically to the other blockstore.
See also
For more details on the MongoDB connection string URI, see Connection String URI Format in the MongoDB Manual.
Delete an S3 Blockstore¶
Navigate to the Snapshot Storage page.¶
- Click the Admin link.
- Click the Backup tab.
- (Optional) If you have not previously set the head directory, set it in the Head Directory box.
- Click the Snapshot Storage page.