For AI agents: a documentation index is available at https://www.mongodb.com/docs/llms.txt — markdown versions of all pages are available by appending .md to any URL path.
Docs Menu
Docs Home
/
Management
/
Ops Manager
Docs Home
/
Management
/
Ops Manager
Docs Home
/
Management
/
Ops Manager

Direct to S3 Backup in Ops Manager

When you enable Direct to S3 Backup, the MongoDB Agent uploads snapshot blocks directly to S3 using pre-signed URLs that Ops Manager provides. Ops Manager handles only the metadata for the upload, such as generating pre-signed URLs, tracking the block manifest, and signaling completion. The snapshot data path bypasses Ops Manager and no longer flows through the Ops Manager server. This removes the proxy bottleneck for large deployments and frequent backup schedules.

This topic explains how to enable Direct to S3 Backup and how to configure its optional settings.

Important

Direct to S3 Backup requires Ops Manager 9.0.0 or later, or Ops Manager 8.0.27 or later, and a minimum MongoDB Agent version on every host that runs the backup:

Ops Manager Version
Minimum MongoDB Agent Version

9.0

109.0.0.9279-1

8.0

108.0.27.9088-1

How Direct to S3 Backup Works

When Direct to S3 Backup is enabled on a backup job and the MongoDB Agent on the deployment meets the minimum version requirement, Ops Manager performs the following actions:

  • Ops Manager generates pre-signed upload URLs using the S3 credentials configured for the snapshot blockstore. The URLs point to the same S3 bucket that the blockstore uses.

  • Ops Manager sets uploadPath=AGENT_DIRECT_S3 in the backup cursor description. The MongoDB Agent reads this value, freezes the upload path for the job, and uploads snapshot blocks directly to S3 using those pre-signed URLs.

  • Ops Manager tracks upload completion through the block manifest APIs. No /dataBlocks proxy upload calls occur.

Because the MongoDB Agent uploads snapshot data directly to S3, Ops Manager no longer sits in the data path for backups. Ops Manager remains responsible for backup metadata only.

Coexistence and Fallback

Direct to S3 Backup and standard backups that stream data through Ops Manager can coexist.

On a job where you enable Direct to S3 Backup, Ops Manager falls back to the standard path when the MongoDB Agent version is below the minimum. The fallback lasts until you upgrade the MongoDB Agent. Mixed-version fleets continue to back up successfully:

  • Deployments that meet the version requirement use Direct to S3 Backup.

  • Deployments that do not meet the version requirement use the standard data path. The job automatically switches to Direct to S3 Backup on the first snapshot after you upgrade the MongoDB Agent, with no further action required. A snapshot that is already in progress completes on its original upload path.

Additionally, you can manually enable or disable Direct to S3 Backup for a job, as described below.

Note

Direct to S3 Backup applies only to snapshot block data. Oplog data for point-in-time recovery continues to use the standard backup path through the configured oplog store.

Prerequisites

Before you enable Direct to S3 Backup:

  • Ensure you are running Ops Manager 9.0.0 or later, or Ops Manager 8.0.27 or later.

  • Ensure a MongoDB Agent version of at least 109.0.0.9279-1 for Ops Manager 9.0 deployments, or 108.0.27.9088-1 for Ops Manager 8.0 deployments, on every host that runs the backup.

  • Ensure the deployment has an S3 snapshot store (blockstore). Ops Manager uses the same S3 bucket and credentials that the blockstore uses.

Backup Host Network Access

Each host that runs a backup must be able to:

  • Resolve the DNS name of the S3 or S3-compatible endpoint

  • Establish outbound HTTPS connections to that endpoint

  • Upload data using the S3 pre-signed URLs that Ops Manager generates

When you enable Direct to S3 Backup, the high-volume upload path moves from the Ops Manager application servers to every host that runs a MongoDB Agent. Previously, only the Ops Manager servers needed network access to S3. Verify outbound connectivity from a backup host to the S3 endpoint before you enable the feature.

MongoDB Agents do not require S3 access keys or IAM roles. Ops Manager generates all pre-signed URLs using the S3 credentials configured for the snapshot store. Those credentials must permit pre-signed PUT operations on the bucket, not only reads.

Enable Direct to S3 Backup

To enable Direct to S3 Backup, complete the following steps.

1

Enable the Direct to S3 Backup feature flag.

  1. In the Ops Manager Admin console, click General and Ops Manager Config.

  2. Click the Custom tab.

  3. Add one of the following key and value pairs to enable Direct to S3 Backup at the global or project level:

    Access Level
    Key
    Value

    Project

    mms.featureFlag.backup.d2s3

    controlled

    Global

    mms.featureFlag.backup.d2s3

    enabled

  4. Click Save.

2

(Conditional) Enable Direct to S3 Backup in the project.

If you set the flag to controlled in the previous step, enable the feature in your project settings:

  1. In your Ops Manager project, click Settings.

  2. Click the Beta Features tab, and click Backup D2s3.

3

(Conditional) Enable Direct to S3 Backup on existing backup jobs.

In a project where you enabled Direct to S3 Backup, new backup jobs start with Direct to S3 Backup already enabled. The first snapshot of each new job uploads directly to S3, and no per-job action is required.

Backup jobs that existed before you enabled the feature keep their existing setting. You can also enable or disable Direct to S3 Backup on any job. To change the setting for a job:

  1. Click Admin, Backup, then Jobs.

  2. For the target job, locate the Direct S3 Backup row. The row appears only when the job's snapshot store is an S3 blockstore.

  3. Set Direct S3 Backup to Enabled or Disabled.

  4. For sharded clusters, select Apply to all cluster members to enable the feature on every shard and the config server.

  5. Click Save.

4

Verify the agent version on every host.

Check the version of every MongoDB Agent that runs the backup and ensure it meets the minimum version for your Ops Manager deployment.

The following table summarizes when a backup job uses Direct to S3 Backup:

Scenario
Upload path

Job created after you enabled the feature in the project

Direct to S3 Backup, starting with the first snapshot

Job created before you enabled the feature in the project

Standard path until you enable Direct to S3 Backup on the job

Job with Direct to S3 Backup enabled and a MongoDB Agent below the minimum version

Standard path until you upgrade the MongoDB Agent. The job switches to Direct to S3 Backup on the first snapshot after the upgrade.

Manage Optional Settings

The following optional settings tune Direct to S3 Backup upload performance.

mms.backup.d2s3.transfer.numWorkers

Number of parallel upload workers that the MongoDB Agent uses for Direct to S3 Backup uploads. Default: 2. You can also set a per-job override.

Configure this setting in the Ops Manager configuration file alongside the other mms.backup.* properties. If you set this property, the MongoDB Agent uses that exact number of workers and skips its own auto-tuning. If you leave it unset, the MongoDB Agent sizes the worker count based on the host's CPU and memory.

mms.backup.d2s3.transfer.maxNumUnitOfWorkBlocks

Maximum number of blocks per work unit for the Direct to S3 upload path. When unset, the MongoDB Agent falls back to an internal default of 100.

Configure this setting using one of the following:

Note

The number of concurrent connections to S3 increases with the number of backup jobs that run at the same time. If your deployment runs many backup jobs concurrently, the total number of S3 connections may be high.

Operational Considerations

Performance and Sizing

  • Ops Manager load: Ops Manager application servers no longer carry snapshot block payloads, but they still handle pre-signing, manifest validation, job state, and metadata writes. Size Ops Manager for control-plane workloads.

  • Agent host sizing: Direct to S3 Backup moves compression, hashing, TLS, and parallel S3 uploads from Ops Manager servers to each host that runs a MongoDB Agent. Before enabling the feature, make sure these hosts have enough CPU, memory, and outbound network capacity for the additional workload. Capture a baseline snapshot duration and resource usage so you can compare the impact after enabling Direct to S3.

    • When mms.backup.d2s3.transfer.numWorkers is not set, the MongoDB Agent automatically sizes the number of backup workers based on the CPU and memory the host is allowed to use. It starts at roughly one worker per 4 vCPUs of that allowance, up to a maximum of 16 workers. Each worker can use about one CPU core at peak, so plan to keep at least 25% of the host's allowed CPU capacity for backup.

    • During snapshots, monitor MongoDB Agent CPU usage and database latency. If the MongoDB Agent is CPU-bound or database latency increases, consider adding capacity or reducing numWorkers.

  • Network path: Total backup throughput is the aggregate of every MongoDB Agent-to-S3 path. Confirm that your S3 endpoint, VPC endpoint, or proxy can handle the expected concurrency across all hosts backing up simultaneously.

Security and IAM

  • Ops Manager permissions: The S3 credentials configured for the snapshot store must permit pre-signed PUT operations on the bucket and prefixes used for snapshots.

  • MongoDB Agent permissions: MongoDB Agents do not have S3 credentials. Ops Manager generates pre-signed URLs for every block upload and verification.

  • Immutability: Direct to S3 Backup is compatible with S3 Object Lock. Object version IDs required by immutable snapshots are recorded in the block manifest.

Learn More

Back
Immutable Snapshots
Next
Third-Party Platforms
Rate this page

On this page