Export Cloud Backup Snapshot
On this page
Note
This feature is not available for M0
free clusters, M2
, and
M5
clusters. To learn more about which features are unavailable,
see Atlas M0 (Free Cluster), M2, and M5 Limits.
Atlas lets you export your Cloud Backup snapshots to an AWS S3 Bucket or an Azure Blob Storage Container.
To learn how to manage automated backup policies and schedules, see Manage Backup Policies.
How Atlas Exports Snapshots
You can manually export individual snapshots or set up an export policy for automatic export of your snapshots. For automatic exports, you must specify a frequency in your export policy:
Daily
Weekly
Monthly
Yearly
Atlas automatically exports any backup snapshot with the frequency type that matches the export frequency. The exported result is a full backup of that snapshot.
Example
Consider the following:
A backup policy that sets a weekly and monthly snapshot schedule
An export policy that sets a monthly export frequency
Suppose, at the end of the month, the weekly and monthly snapshots
happen on the same day. There would be 4
snapshots of which
3
would be weekly snapshots and the fourth snapshot, although
treated as a weekly snapshot by Atlas, would also be the monthly
snapshot because it happened on the same day. Atlas will export
the monthly snapshot only because the export frequency matches the
snapshot frequency for that snapshot. To export the weekly snapshots
as well, update the export policy to export weekly snapshots also.
If the export frequency is set to weekly, Atlas would export all
4
snapshots.
Atlas exports snapshots for collections one at a time. As the export progresses, you may see partial results in your S3 Bucket or Azure Blob Storage Container. Atlas queues any new job if Atlas is currently exporting 5 or more replica sets. For sharded clusters, Atlas always exports the snapshots of all the shards simultaneously, regardless of the number of shards.
Atlas charges $.125
per GB of data exported to the AWS S3
bucket or Azure Blob Storage Container, in addition to the data transfer cost
incurred from AWS or Azure itself. Atlas compresses the data
before exporting. To estimate the amount of data being exported, add up
the dataSize
of each database in your cluster. This total should correspond to
the uncompressed size of your export, which would be the maximum cost
incurred from Atlas for the data export operation.
Files Atlas Uploads
Atlas uploads an empty file to /exported_snapshots/.permissioncheck
when you:
Add a new AWS S3 Bucket or an Azure Blob Storage Container for export
Start an export
After Atlas finishes exporting, Atlas uploads a metadata file
named .complete
and a metadata file named metadata.json
for
each collection.
Atlas uploads the metadata file named .complete
in the
following path on your S3 Bucket:
/exported_snapshots/<orgUUID>/<projectUUID>/<clusterName>/<initiationDateOfSnapshot>/<timestamp>/
Note
By default, Atlas uses organization and project UUIDs in
the path for the metadata files. To use organization and
project names instead of UUIDs, set the
useOrgAndGroupNamesInExportPrefix
flag to true
via the
API. Atlas
replaces any spaces with underscores (_
) and removes any
characters that might require special handling and
characters to avoid from the
organization and project names in the path.
The .complete
metadata file is in JSON format and contains
the following fields:
Field | Description |
---|---|
orgId | Unique 24-hexadecimal digit string that identifies the
Atlas organization. |
orgName | Name of the Atlas organization. |
groupId | Unique 24-hexadecimal digit string that identifies the
project in the Atlas organization. |
groupName | Name of the Atlas project. |
clusterUniqueId | Unique 24-hexadecimal digit string that identifies the
Atlas cluster. |
clusterName | Name of the Atlas project. |
snapshotInitiationDate | Date when snapshot was taken. |
totalFiles | Total number of files uploaded to the S3 Bucket or
Azure Blob Storage Container. |
labels | Labels of the cluster whose snapshot was exported. |
customData | Custom data, if any, that you specified when creating the
export job. |
Example
{ "orgId": "60512d6f65e4047fe0842095", "orgName": "org1", "groupId": "60512dac65e4047fe084220f", "groupName": "group1", "clusterUniqueId": "60512dac65e4047fe0842212", "clusterName": "cluster0", "snapshotInitiationDate": "2020-04-03T05:50:29.321Z" "totalFiles": 23, "labels": [ { "key": "key1", "value": "xyz" }, { "key": "key2", "value": "xyzuio" } ], "customData": [ { "key": "key1", "value": "xyz" }, { "key": "key2", "value": "xyzuio" } ] }
Atlas uploads the metadata.json
file for each collection
in the following path on your S3 Bucket:
/exported_snapshots/<orgUUID>/<projectUUID>/<clusterName>/<initiationDateOfSnapshot>/<timestamp>/<dbName>/<collectionName>/metadata.json
Note
By default, Atlas uses organization and project UUIDs in
the path for the metadata files. To use organization and
project names instead of UUIDs, set the
useOrgAndGroupNamesInExportPrefix
flag via the
API to true.
Atlas replaces any spaces with underscores (_
) and
removes any characters that might require special
handling and
characters to avoid from the
organization and project names in the path.
The metadata file is in JSON format and contains the following fields:
Field | Description |
---|---|
collectionName | Human-readable label that identifies the collection. |
indexes | List of all the indexes on the collection in the format
returned by db.collection.getIndexes
command. |
options | Configuration options defined on the collection. To learn
more about the options, see db.createCollection() command. |
type | Type of collection. Value can be one of the following:
This field is not returned for a standard collection. |
uuid | Collection's UUID. To learn more about UUID, see
UUID. |
Example
{ "options":{ "viewOn":"othercol", "pipeline":[{"$project":{"namez":"$name"}}] }, "indexes":[], "collectionName":"viewcol", "type":"view" }
{ "options":{ "timeseries":{ "timeField":"timestamp", "granularity":"seconds", "bucketMaxSpanSeconds":{"$numberInt":"3600"} } }, "indexes":[], "collectionName":"timeseriescol", "type":"timeseries" }
{ "indexes": [ { "v":{"$numberInt":"2"}, "key":{ "_id":{"$numberInt":"1"} }, "name":"_id_" } ], "uuid":"342c40a937c34c478bab03de8ce44f3e", "collectionName":"somecol" }
If an export job fails:
Atlas doesn't automatically try to export again.
Atlas doesn't remove any partial data in your S3 Bucket or Azure Blob Storage Container.
Exported Data Format
Atlas uploads the contents of your database to S3 or Azure Blob Storage in
.json.gz
format with documents in extended JSON format within. The
following is the path to the files on S3 or Azure Blob Storage:
/exported_snapshots/<orgName>/<projectName>/<clusterName>/<initiationDateOfSnapshot>/<timestamp>/<dbName>/<collectionName>/<shardName>.<increment>.json.gz
Where:
<orgName> | Name of your Atlas organization. |
<projectName> | Name of your Atlas project. |
<clusterName> | Name of your Atlas cluster. |
<initiationDateOfSnapshot> | Date when snapshot was taken. |
<timestamp> | Timestamp when the export job was created. |
<dbName> | Name of the database in the Atlas cluster. |
<collectionName> | Name of the Atlas collection. |
<shardName> | Name of the replica set. |
<increment> | Count that is incremented as chunks are uploaded. Starts at
0 . |
Limitations
The following limitations apply:
You can only export snapshots for currently supported versions of MongoDB, but you can always download saved snapshots, regardless of the version.
You can't export fallback snapshots.
You can have only one active export per snapshot.
Required Access
To manage your Cloud Backup snapshots, you must have
Project Owner
access to the project. Users with
Organization Owner
access must add themselves as a Project
Owner
to the project before they can manage Cloud Backup
snapshots.
Prerequisites
To export your Cloud Backup snapshots, you need an M10
or higher
Atlas cluster with Cloud Backup enabled. In addition, to export to an AWS S3
Bucket or Azure Blob Storage Container, you must do the following:
Configure AWS IAM role with
STS:AssumeRole
that grants Atlas access to your AWS resources. To learn more about configuring AWS access for Atlas, see Set Up Unified AWS Access.Configure AWS IAM role policy that grants Atlas write access or the
S3:PutObject
andS3:GetBucketLocation
permissions to your AWS resources. To learn more about configuring write access to AWS resources, see Set Up Unified AWS Access.Example
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:GetBucketLocation", "Resource": "arn:aws:s3:::bucket-name" }, { "Effect": "Allow", "Action": "s3:PutObject", "Resource": "arn:aws:s3:::bucket-name/*" } ] }
Set up Azure Service Principal with access policy for your Atlas project.
Assign the Storage Blob Delegator and Storage Blob Data Contributor roles to your Azure Service Principal.
To assign the roles to your Service Principal, you will need the following information:
RoleDescriptionStorage Blob DelegatorThis allows the Service Principal to sign SAS tokens to access the Azure Storage Container. To assign this role, run the following command:
az role assignment create --assignee-object-id <service-principal-id> --role "Storage Blob Delegator" --scope /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Storage/storageAccounts/<storage-account-name> Storage Blob Data ContributorThis allows read, write, and delete blob access for the Azure Storage Container. To assign this role, run the following command:
az role assignment create --assignee-principal-type ServicePrincipal --assignee-object-id <service-principal-id> --role "Storage Blob Data Contributor" --scope /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Storage/storageAccounts/<storage-account-name>/blobServices/default/containers/<container-name>
Export Management
You can create and manage snapshot exports to AWS S3 Buckets from the Atlas CLI and Atlas Administration API or to Azure Blob Storage Containers from the Atlas Administration API.
Note
You can't export snapshots to Azure Blob Storage Containers using the Atlas CLI.
Manage Export Jobs
You can manage export jobs using the Atlas CLI by creating or viewing export jobs.
Create an Export Job
To export one backup snapshot for an M10 or higher Atlas cluster to an existing AWS S3 Bucket using the Atlas CLI, run the following command:
atlas backups exports jobs create [options]
To watch for a specific backup export job to complete using the Atlas CLI, run the following command:
atlas backups exports jobs watch <exportJobId> [options]
To learn more about the syntax and parameters for the previous commands, see the Atlas CLI documentation for atlas backups exports jobs create and atlas backups exports jobs watch.
View Export Jobs
To list the cloud backup restore jobs for the project you specify using the Atlas CLI, run the following command:
atlas backups exports jobs list <clusterName> [options]
To return the details for the cloud backup restore job you specify using the Atlas CLI, run the following command:
atlas backups exports jobs describe [options]
To learn more about the syntax and parameters for the previous commands, see the Atlas CLI documentation for atlas backups exports jobs list and atlas backups exports jobs describe.
Manage Export Buckets
You can manage export buckets using the Atlas CLI by creating, viewing, or deleting export buckets.
Create One Export Bucket
To create an export destination for Atlas backups using an existing AWS S3 bucket using the Atlas CLI, run the following command:
atlas backups exports buckets create <bucketName> [options]
To learn more about the command syntax and parameters, see the Atlas CLI documentation for atlas backups exports buckets create.
View Export Buckets
To list the cloud backup restore buckets for the project you specify using the Atlas CLI, run the following command:
atlas backups exports buckets list [options]
To return the details for the cloud backup restore bucket you specify using the Atlas CLI, run the following command:
atlas backups exports buckets describe [options]
To learn more about the syntax and parameters for the previous commands, see the Atlas CLI documentation for atlas backups exports buckets list and atlas backups exports buckets describe.
Delete an Export Bucket
To delete an export destination for Atlas backups using the Atlas CLI, run the following command:
atlas backups exports buckets delete [options]
To learn more about the command syntax and parameters, see the Atlas CLI documentation for atlas backups exports buckets delete.
To grant and manage cloud provider access and to create and manage
snapshot export jobs, the API that you use must have the
Project Owner
role.
Manage Export Buckets
Use the following to manage export Buckets or Containers.
Create One Export Bucket
To grant access to AWS S3 Bucket or Azure Blob Storage Container for
exporting snapshots, send a POST
request to the
Cloud Backups resource
endpoint. This enables the
AWS S3 Bucket or Azure Blob Storage Container to receive Atlas
Cloud Backup snapshots. When sending the request to grant access,
you must provide the following information:
Unique 24-hexadecimal character string that identifies the unified AWS access role ID that Atlas must use to access the AWS S3 Bucket. To learn more, see Set Up Unified AWS Access.
Unique 24-hexadecimal character string that identifies the Azure Service Principal that Atlas must use to access the Azure Blob Storage. To learn more, see Set Up and Manage Azure Service Principal Access.
Service endpoint of your Azure Blob Storage account. To learn more, see Azure Documentation.
UUID string that identifies the Microsoft Entra tenant ID. To learn more, see Microsoft Entra Documentation.
List All Export Buckets
To retrieve all the AWS S3 Buckets and Azure Blob Storage Containers
to which Atlas exports snapshots, send a GET
request to
the Cloud Backups resource
endpoint.
Delete One Export Bucket
To delete an Export Bucket, you must first disable automatic export of snapshots to
the AWS S3 Bucket or Azure Blob Storage Container for all clusters
in the project and then send a DELETE
request to the
Cloud Backups resource
endpoint with the ID of the
Export Bucket. If necessary, send a GET
request to the
endpoint to retrieve the
export Bucket ID.
Manage Export Jobs
Use the following to manage export jobs.
Create Snapshot Export Job
To export one Atlas backup snapshot to an AWS S3 Bucket or
Azure Blob Storage Container, send a POST
request to the
Cloud Backups resource
endpoint with the ID of
the snapshot to export and the ID of the AWS S3 Bucket or
Azure Blob Storage Container.
Retrieve Snapshot Export Job
To retrieve one snapshot export job by its ID, send a GET
request to the Cloud Backups
resource endpoint with the
ID of the export job.
To retrieve all running snapshot export jobs, send a GET
request to the Cloud Backups
resource endpoint.