For AI agents: a documentation index is available at https://www.mongodb.com/docs/llms.txt — markdown versions of all pages are available by appending .md to any URL path.

Portable S3 Backups with Object Lock

Ops Manager can store backups, including snapshots and oplogs, in an S3 or S3-compatible object store by writing backup data as blocks to an S3 bucket and tracking metadata in the Ops Manager application database. With portable S3 backups, Ops Manager integrates with S3 Object Lock or an equivalent to provide the following capabilities:

  • Store backup data and backup metadata together in the S3 bucket.

  • Protect backup objects from deletion or modification until a defined retention period expires, enforced by the object store.

  • Write portable restore metadata copies into the bucket, so Ops Manager can import a full point-in-time restore chain, snapshot and oplog together, into a new or rebuilt Ops Manager deployment for disaster recovery or migration.

Use this feature to protect backups against malicious or accidental deletion, such as ransomware attacks or operator error. You can restore backups in a new Ops Manager instance using only what you store in S3.

This guide covers portable S3 backups for Ops Manager deployments that you install and manage directly on a host. If you deploy Ops Manager using the MongoDB Controllers for Kubernetes Operator (MCK), see the Ops Manager resource specification example.

Important

Ops Manager supports immutable snapshots on Object-Lock-enabled buckets starting in version 8.0.19. Immutable oplogs, which extend this protection to the entire point-in-time recovery chain, require version 9.0 or 8.0.28. Portable backups, which add portable restore metadata so you can import a full backup from only what the S3 bucket stores, require version 9.0.2 or 8.0.28.

After you configure an S3 snapshot store with an Object-Lock-enabled bucket and enable portable backups:

  • Ops Manager writes snapshot blocks and the metadata required for restore into the S3 bucket, instead of keeping metadata only in the Ops Manager application database.

  • If configured, Ops Manager extends the Object Lock retention for each snapshot so that the object store prevents deletion or modification before the backup job retention window ends.

  • Ops Manager can discover and import these immutable snapshots from a pre-populated bucket into another Ops Manager instance by reading only what the S3 bucket stores.

If you also configure an S3 oplog store with Object Lock enabled:

  • Ops Manager writes oplog slices and the metadata required for restore into the S3 oplog store's bucket, the same way it does for snapshot data.

  • The MongoDB Agent tails only majority-committed oplog entries for that backup job, so Ops Manager never backs up an oplog entry that a later rollback could remove.

  • Ops Manager can discover and import these immutable oplogs, together with an imported immutable snapshot, into another Ops Manager instance to support point-in-time restore from the imported backup.

A portable backup is a backup that Ops Manager can import into another Ops Manager deployment using only the data stored in the S3 bucket. To make a backup portable, Ops Manager writes portable restore metadata copies of the backup's snapshot or oplog data into the bucket alongside the backup data.

Portability is a choice you make for each store:

  • On an S3 snapshot store, select Portable Snapshot to write portable restore metadata for snapshots.

  • On an S3 oplog store, select Portable Oplog to write portable restore metadata for oplog slices.

Portable backups require an Object-Lock-enabled bucket. Ops Manager rejects the portable choice on a store whose bucket does not have Object Lock enabled.

Important

You cannot disable portability after you enable it on a store.

Coverage is forward-looking. Ops Manager writes portable restore metadata copies only for backups it takes after you enable the choice on a store. Snapshots taken before you enable portability on the snapshot store never receive metadata copies and are not importable, even after the option is on. On-demand snapshots taken after enablement are fully covered.

Warning

A bucket can have Object Lock enabled without a default retention rule. Data written to such a bucket is versioned but not protected, because objects can still be deleted. If you select the portable choice on a store backed by such a bucket, Ops Manager warns you that nothing written to the store is protected by any retention until you set a default retention rule on the bucket. Setting the rule afterwards is safe. Protection then applies to everything written after the rule is in place.

When you create an S3 snapshot store or S3 oplog store on an Object-Lock-enabled bucket, Ops Manager reads the bucket's Object Lock settings and stores them in the store's configuration. You do not enter the Object Lock retention mode or period in Ops Manager. The Object Lock Enabled, Object Retention Mode, and Object Retention Period (Days) fields display the ingested values and are read-only.

If a request to the S3 store APIs supplies Object Lock retention values, Ops Manager discards them, logs a warning, and continues the operation. The values ingested from the bucket are the only ones Ops Manager stores.

Ops Manager refuses to create the store if it cannot determine the bucket's Object Lock state.

A background job realigns each saved store's configuration with its bucket every hour. This job is the only mechanism that changes a store's ingested retention mode and period. If the bucket's settings change, Ops Manager updates the store's configuration to match. The job never changes the portable choice you made for the store.

To use portable S3 backups, ensure you have the following:

  • Ops Manager 8.0.19 or later for immutable snapshots. Portable backups additionally require version 9.0.2 or 8.0.28.

  • S3 snapshot store (blockstore) configured in Ops Manager. Immutable snapshots support only S3-based snapshot stores, not MongoDB blockstores or filesystem snapshot stores.

  • Object store that supports Object Lock, such as AWS S3 or compatible appliances, with Object Lock enabled on the bucket you use for the store. Most S3 providers require Object versioning for Object Lock to function, and some providers allow Object Lock only during bucket creation.

  • Standard backup requirements, such as a MongoDB Agent with backup enabled on suitable nodes and an oplog (operations log) store for PITR.

  • An S3 bucket with Object Lock enabled for your snapshot store. The bucket doesn't need to be new, and doesn't need to be dedicated to this store, though a dedicated bucket is recommended. You can create a new S3 snapshot store that points to the bucket, or enable Object Lock on the bucket of an existing store and let Ops Manager detect the change.

  • If you use portable backups: an S3 bucket with versioning and Object Lock enabled for your oplog store. The bucket doesn't need to be new, and doesn't need to be dedicated to this store, though a dedicated bucket is recommended. You can create a new S3 oplog store that points to the bucket, or enable versioning and Object Lock on the bucket of an existing store and let Ops Manager detect the change.

When you plan portable S3 backups, consider the following items:

  • No retroactive protection. Ops Manager protects only snapshots written to a bucket with Object Lock after you enable immutable backup. Ops Manager does not apply Object Lock to existing snapshot objects.

  • Object Lock configuration. Ops Manager does not provide immutability for S3 buckets that do not support Object Lock. You must configure Object Lock on the bucket separately.

  • No automatic migration of existing data. The Migrate Backup Between S3 Stores feature lets you change which S3 snapshot store a backup job uses for future snapshots. It does not copy or move existing snapshot data between buckets.

  • Retention and Object Lock on managed stores. When a snapshot ages out according to the backup job retention policy, Ops Manager stops exposing it and attempts to delete the related objects. Ops Manager can delete the objects only after the Object Lock retention period expires. If the object store rejects a delete, for example because an object is still under retention, the groom job does not fail. Ops Manager queues the objects that it can't delete, and the Backup Daemon retries the deletion every mms.backup.groom.failedDeleteRetryDelayHours hours until the object store allows them.

  • Retention on imported stores. Ops Manager never deletes data from the S3 bucket you imported from. Ops Manager can delete the metadata it imported into its own application database. Retention affects only metadata visibility in Ops Manager.

  • Variable retention mode is not supported. Use buckets with a fixed default retention mode and period. Ops Manager does not detect or validate variable retention configurations on the bucket.

  • Majority-commit topology requirement for oplog tailing. Ops Manager only tails majority-committed oplog entries for an Object-Lock-enabled oplog store, since Object Lock means Ops Manager can never undo or change data already written to the store. On a PSA topology with the secondary down, or a PAA topology, the oplog tail can never reach majority commit, so tailing stalls until the topology recovers. Ops Manager raises an alert naming the cause and the remedy when this happens.

To configure portable snapshot storage, complete the following tasks:

  1. Enable portable backups in Ops Manager.

  2. Prepare an Object-Lock-enabled S3 bucket.

  3. Configure a new S3 snapshot store in Ops Manager.

An application setting controls portable backups. A global administrator must enable this setting before you can select the portable option on a store backed by an Object-Lock-enabled bucket.

1

Click Admin, General, Ops Manager Config.

2
  1. Click the Custom tab.

  2. Click + Add Option.

  3. In the Key field, enter brs.immutableBackupEnabled.

  4. In the Value field, enter true.

  5. Click Save.

This setting enables portable backups on stores backed by Object-Lock-enabled buckets. Ops Manager applies this setting immediately. You do not need to restart Ops Manager.

Note

You can also set brs.immutableBackupEnabled to true in conf-mms.properties instead of using the Admin UI. If you set the setting in both places, Ops Manager uses the conf-mms.properties value.

1

You can use an existing S3 or S3-compatible bucket. Ops Manager doesn't require a bucket dedicated to portable MongoDB backups, though we recommend using one.

2

Enable Object Lock on the bucket. Depending on your provider, you can do this during bucket creation or afterward. Most S3 providers require Object versioning for Object Lock to work, and some providers allow Object Lock only during bucket creation.

3

Ops Manager uses your bucket's conditional write support by default to reduce the storage space that portable snapshot metadata uses. Configure your bucket to support conditional writes.

Important

We recommend using conditional write support, but Ops Manager does not require it. If your bucket does not support conditional writes, Ops Manager still protects your snapshots with Object Lock, but the bucket can grow at an unexpected rate.

4

Configure a default retention mode (GOVERNANCE or COMPLIANCE) and period on the bucket. Ops Manager ingests these settings from the bucket when you create the store. Without a default retention configured on the bucket, Object Lock does not protect your objects from deletion or modification.

Note

If you already use an S3 bucket for backups but need portable snapshots, enable Object Lock on that bucket if your provider allows it, or point to a different bucket that already has Object Lock enabled. Ops Manager ingests the bucket's settings when you create the store.

1

Follow the Add One S3-Compatible Snapshot Store procedure to create a new S3 snapshot store that points to the Object-Lock-enabled bucket. Specify the bucket name, endpoint, region, and credentials as required by your S3 provider.

2

Ops Manager reads the bucket's Object Lock settings when you create the store. The Object Lock Enabled, Object Retention Mode, and Object Retention Period (Days) fields display the ingested values and are read-only.

3

Select Portable Snapshot to write portable restore metadata for snapshots.

4

Ops Manager automatically validates connectivity when you save the store.

Important

You cannot change the Portable Snapshot choice after you save the store.

This snapshot store serves as the target for portable snapshots. Ops Manager writes snapshot blocks, metadata, and portable restore metadata copies for backups that use this store into the bucket.

Note

If you manage Ops Manager with the MongoDB Enterprise Kubernetes Operator, you must also enable Object Lock on the corresponding s3Stores entry in your MongoDBOpsManager resource. Set the spec.backup.s3Stores[].objectLockEnabled field to true so the Kubernetes Operator reconciles Ops Manager with Object-Lock awareness.

You still need a MongoDB instance to act as a metadata store for Ops Manager operations.

To configure portable oplog storage, complete the following tasks:

  1. Enable portable backups in Ops Manager, if you haven't already.

  2. Prepare a versioned, Object-Lock-enabled S3 bucket.

  3. Configure a new S3 oplog store in Ops Manager.

  4. Switch existing backup jobs to the portable S3 oplog store.

This step uses the same Ops Manager application setting as portable snapshot storage. If you've already completed Enable Portable Backups while configuring portable snapshot storage, skip this step.

Prepare a bucket for your portable oplog store.

1

You can use an existing S3 or S3-compatible bucket. Ops Manager doesn't require a bucket dedicated to portable MongoDB backups, though we recommend using one.

2

Unlike a standard oplog store, a portable oplog store requires both versioning and Object Lock enabled on the bucket. Depending on your provider, you can enable these during bucket creation or afterward.

3

Ops Manager uses your bucket's conditional write support by default to reduce the storage space that portable backup metadata uses. Configure your bucket to support conditional writes.

Important

We recommend using conditional write support, but Ops Manager does not require it. If your bucket does not support conditional writes, Ops Manager still protects your oplogs with Object Lock, but the bucket can grow at an unexpected rate.

4

Configure a default retention mode (GOVERNANCE or COMPLIANCE) and period on the bucket. Ops Manager ingests these settings from the bucket when you create the store. Without a default retention configured on the bucket, Object Lock does not protect your objects from deletion or modification.

Note

If you already use an S3 bucket for oplog storage but need portable backups, enable versioning and Object Lock on that bucket if your provider allows it, or point to a different bucket that already has both enabled. Ops Manager ingests the bucket's settings when you create the store.

Configure the new bucket as an S3 oplog store.

1

Follow the Add One S3-Compatible Oplog Store procedure to create a new S3 oplog store that points to the versioned, Object-Lock-enabled bucket. Specify the bucket name, endpoint, region, and credentials as required by your S3 provider.

2

Ops Manager reads the bucket's Object Lock settings when you create the store. The Object Lock Enabled, Object Retention Mode, and Object Retention Period (Days) fields display the ingested values and are read-only.

3

Select Portable Oplog to write portable restore metadata for oplog slices. You cannot disable this choice after you save the store.

4

Ops Manager automatically validates connectivity when you save the store.

Important

You cannot change the Portable Oplog choice after you save the store.

This oplog store serves as the target for portable backups. Ops Manager writes oplog slices, metadata, and portable restore metadata copies for backups that use this store into the bucket.

If your backup jobs already use an S3 store, you can make that store portable without creating a new one:

1

Enable Object Lock on the S3 bucket that backs the store. If your provider allows it, also enable versioning and set a default retention rule on the bucket.

2

Ops Manager realigns store configurations with their buckets hourly. After the realignment, the store's Object Lock fields display the values ingested from the bucket.

3

Follow the Edit One Existing S3-Compatible Oplog Store procedure and select Portable Oplog. Ops Manager writes portable restore metadata for that store's oplog backups. You cannot disable this choice afterwards.

Important

Enable portability on the oplog store before you enable it on the snapshot store, or at the same time. Never enable the snapshot store first.

Ops Manager writes oplog metadata copies only when it writes an oplog slice. There is no backfill for slices written while the choice was off. If you enable portability on the snapshot store first, snapshots taken in the minutes before the oplog store follows become importable while their point-in-time restore window is incomplete. A restore anchored on one of them fails when it tries to replay oplog from that period.

If you enabled the stores in the wrong order, complete the following steps. After you finish, every restore anchor is sound:

1

This completes the pair of portable stores.

2

Ops Manager covers this snapshot with complete oplog metadata from its own timestamp forward.

3

Until they expire from the retention schedule, do not use snapshots taken between the two enablements as point-in-time restore anchors. Plain snapshot restores from them still work. Only point-in-time restores are affected.

To reassign an existing backup job to a portable S3 oplog store, use assignment labels. This method is a fallback. To keep the existing oplog store, follow the steps in Enable Portability on an Existing Store instead.

1

Follow the Edit One Existing S3-Compatible Oplog Store procedure to add a label to the portable S3 oplog store.

2
  1. On the Jobs page, click the project name to open the project's assignment page.

  2. In the Assignment Labels list box, click Select Labels.

  3. Select the label.

  4. Click Save Changes.

3

Ops Manager reassigns the backup job's oplog store to match the labels only after you restart backups.

Important

PIT restores are unavailable until Ops Manager takes the next snapshot after the reassignment.

Ops Manager automatically writes each snapshot's data and the Object Lock metadata that protects it to the S3 bucket. You do not need to manually export the data. Use the following procedure to confirm that a specific snapshot is protected by Object Lock.

1

Go to the View All Snapshots page for the backup you want to check, then find the snapshot you want to confirm in the list.

2

Examine the Object Lock column for that snapshot. If the column displays Yes, Ops Manager manages the snapshot as immutable.

To confirm that an S3 oplog store has Object Lock enabled, go to the Oplog Storage page and check that store's settings.

If you need to restore data from a pre-existing S3 bucket that contains portable snapshots and backup metadata, you can import those snapshots into an Ops Manager deployment. Use this workflow for disaster recovery scenarios, such as restoring backups after rebuilding Ops Manager.

Important

Imported snapshots are for one-time restore operations only. Ops Manager does not manage the lifecycle of imported snapshots or include them in active backup schedules.

1

Click Admin, Backup, Imported Snapshots.

2

Click Import Snapshot in the upper right corner of the page to open the import wizard.

3

Select an existing bucket, or select Add S3 bucket to define a new one. The bucket name and S3 endpoint must match the values configured on the snapshot's S3 store.

Important

The Add S3 bucket form asks for the same fields as the Add One S3-Compatible Snapshot Store procedure, including a datastore type and MongoDB host list or URI. These fields must point to the dedicated metadata store for this bucket, not to the MongoDB deployment that you are restoring.

4

Select the snapshots or clustershots that you want to import from the bucket.

Important

For a clustershot with shards stored in different buckets, import each shard's bucket separately. Add each additional bucket and select that bucket's shards, one bucket at a time, until you import all of the clustershot's shards.

Ops Manager does not make the clustershot restorable, and does not display it on the Continuous Backup page, until you import all of its shards.

5

Select the organization and project to import the snapshots into. This selection determines which project can restore from the imported snapshots.

Once the import completes, the imported snapshots appear on the Continuous Backup page under the project you assigned them to. Ops Manager displays only clustershots that are fully imported.

Ops Manager never deletes or grooms data from the S3 bucket you imported from. You control the lifecycle of imported snapshot objects at the object storage layer. Ops Manager can delete the metadata it imported into its own application database.

If you need point-in-time restore data from a pre-existing S3 bucket that contains portable oplogs, you can import those oplogs into an Ops Manager deployment. You must import the corresponding snapshot first. Ops Manager can only discover oplogs for a cluster that already has an imported snapshot.

Important

If a metadata file listed in the bucket is missing when Ops Manager tries to import it, the import job fails. Listing the bucket's importable oplogs skips the missing file and logs a warning, but the import job itself reports FAILED.

1
  1. Go to the Imported Snapshots page (Admin, Backup, Imported Snapshots).

  2. For the row of the snapshot you already imported, click the ellipsis (Actions) menu.

  3. Click Import Oplogs.

This opens the Import Oplogs dialog on the S3 Buckets tab.

2

Select an existing bucket from the Select an S3 bucket list, then click Next. Or select Add S3 bucket to register a new one, then click Save to register the bucket and continue.

Important

The Add S3 bucket form asks for the same fields as the Add One S3-Compatible Snapshot Store procedure, including a datastore type and MongoDB URI.

You must also select the checkbox that acknowledges MongoDB doesn't support issues caused by your S3-compatible storage solution before Ops Manager saves the bucket.

3

On the Import Oplog tab, select the oplogs you want to import. Ops Manager disables any oplog it already imported. Click Finish to start the import.

Once the import completes, Ops Manager displays a Manage Oplogs action on that row, where you can view or delete the imported oplogs.

PITR combines a baseline snapshot with oplog data stored in an oplog store. Portable snapshots affect only the baseline snapshot component:

  • When you switch to a portable S3 snapshot store, PITR continues to work from existing snapshots and oplog data. Old snapshots remain in their original store, and the oplog store configuration does not change.

  • After the first full portable snapshot completes on the new store, PITR can use that snapshot as the baseline and continue tailing oplogs from your configured oplog store.

  • PITR from an imported snapshot is available only if you also import the corresponding portable oplogs. Otherwise, you can restore an imported snapshot only to the exact point when Ops Manager created the snapshot.

Important

For full portability protection, use an Object-Lock-enabled oplog store together with an Object-Lock-enabled snapshot store. By default, Ops Manager doesn't enforce this pairing: a backup job can use a portable snapshot store with a standard oplog store, and PITR still works from that combination. To require the pairing, a global administrator must enable the mms.backup.enforceObjectLockOplogsOnObjectLockSnapshot application setting. Ops Manager disables this setting by default. To create an Object-Lock-enabled oplog store, follow the Add One S3-Compatible Oplog Store procedure.

When you enable Object Lock on an oplog store, the MongoDB Agent tails only majority-committed oplog entries for that backup job. A majority-committed entry cannot be rolled back, so this behavior prevents Ops Manager from backing up an oplog entry that a later rollback could remove.

Portable S3 snapshots do not introduce additional data-loss scenarios when you switch snapshot stores. Each snapshot restores from a single store, so restore logic does not require data from multiple buckets for one snapshot.

  • All existing snapshots remain in the original S3 store and are fully restorable.

  • PITR works from the most recent completed snapshot and oplog data.

  • Ops Manager marks the in-progress snapshot to the new store as failed or incomplete and does not use it for restores.

  • The last successful snapshot in the original store remains the latest restorable snapshot.

  • After Ops Manager and the cluster recover, Ops Manager schedules a new full snapshot to the portable store at the next snapshot window.

Because the first snapshot on the new store is always a full snapshot and each snapshot restores from a single store, restore logic does not depend on mixing data from multiple S3 buckets for a single snapshot.