Migrating to Atlas from AWS

Migrating a self-managed MongoDB replica set on AWS to MongoDB Atlas

One page with everything you need to migrate a replica set running in AWS to MongoDB’s fully managed cloud database, MongoDB Atlas, with virtually no downtime.
The MongoDB Atlas Live Migration Service helps you migrate MongoDB databases to our fully managed cloud database, MongoDB Atlas, quickly and securely. It works by connecting to your existing MongoDB database, which we will call the source cluster, and synchronizing it with a cluster running in Atlas, which we will refer to as the target cluster, all while your application continues to function normally. Once the data between the two clusters has been synchronized, you can simply update the database connection string in your application to cut over to your cluster in Atlas.
This guide will walk you through how to use the Atlas Live Migration Service.
Step 1: Review your source cluster
The first step in the process is to review your source cluster to ensure it meets a few basic criteria. If for any reason it doesn’t, you can follow the instructions linked below on how to address open issues before beginning the migration process:
  1. 1.
    Your data is currently in a MongoDB database. This guide focuses on migrating to Atlas from an existing MongoDB deployment. If you have data in other database systems (MySQL, PostgreSQL, DynamoDB, etc), please contact us for help with your migration.
  2. 2.
    Your current MongoDB database is running MongoDB Version 2.6 or greater. Atlas supports the latest versions of MongoDB (3.4, 3.6, and 4.0). If you’re running MongoDB version 2.6 or greater, Atlas Live Migration Service can move your data directly into a newer database version. Ensure you update your MongoDB drivers (found here) and make any necessary code changes at the application level to ensure compatibility, prior to going live in production. If you’re running a version released before 2.6, you can review our docs to see how to upgrade to 2.6.
  3. 3.
    Your current deployment is a MongoDB Replica Set. If your deployment is currently a standalone instance, please convert it to a replica set first.
The migration process requires that authentication is enabled on your source cluster, as this is a security best practice that cannot be switched off in MongoDB Atlas. If you currently do not have authentication turned on, please review the docs on enabling it.
You can verify that authentication is enabled on your source cluster using the mongo command in the shell.
$ mongo mongodb://ec2-34-227-18-13.compute-1.amazonaws.com:27017/ -u  -p --authenticationDatabase admin
MongoDB shell version v3.6.3
Enter password:
connecting to: mongodb://ec2-34-227-18-13.compute-1.amazonaws.com:27017/
MongoDB server version: 3.6.3
testRS:PRIMARY>
And finally, the database user from your source cluster that you will be using to perform the migration (and passing to the Atlas Live Migration Service) must have both the clusterMonitor and backup roles as Atlas needs to be able to read all databases, collections, and the oplog.
To verify that the database user you’re intending to use for the migration has the appropriate roles, run the db.getUser() command:
testRS:PRIMARY> use admin
switched to db admin
testRS:PRIMARY> db.getUser("admin")
{
  "_id" : "admin.admin",
  "user" : "admin",
  "db" : "admin",
  "roles" : [
    {
      "role" : "backup",
      "db" : "admin"
    },
    {
      "role" : "clusterMonitor",
      "db" : "admin"
    }
  ]
}
If you don’t have these roles set up for your database user, you can find more details on how to do so in the documentation.
Step 2: Launch your target cluster in MongoDB Atlas
If you don’t already have a MongoDB Atlas account, you can sign up here. The first screen you’ll see when you’re logged in is the Atlas Cluster Builder, which allows you to choose your preferred cloud provider and the cloud region closest to your application instances.
Next you’ll select your cluster size. Note that the target cluster in the live migration process outlined in this guide must be a Dedicated Cluster. For development or staging environments, deploy an M10 cluster or larger; for production workloads, select at least an M30.

The target cluster in a Live Migration to MongoDB Atlas must be an M10 or larger.

For a more detailed description of how to deploy an Atlas cluster, see the documentation.
Step 3: Open Atlas Live Migration Service
In the Atlas UI, locate the target cluster you’ve deployed. For this guide, we'll be using a cluster named “LiveMigration”.
In the cluster card (shown below) associated with your target cluster, click the ellipsis (...) button and then select “Migrate Data to this Cluster”.
A modal with a high-level overview of the Live Migration Process will appear. This guide will walk through each step in greater detail. Click “I’m ready to migrate”.
Step 4: Whitelist Atlas Live Migration Service on Your Source Cluster
These IP address ranges can be found at the top of the updated modal after you’ve clicked “I’m ready to migrate”.
AWS EC2 servers are protected from unauthorized network access using Security Groups. To whitelist new IP address ranges, either create a new Security Group, or modify your existing Security Group to permit inbound network access from the IP addresses listed above.
Here is an example security group that grants access to the Atlas Live Migration Service.
For additional information on creating or modifying Security Groups, consult the AWS Documentation.
Once your security group has been created or modified, you’ll need to associate this Security Group to the EC2 instances where your source cluster is running. To verify, or change this configuration, visit the EC2 control panel and review the security groups assigned to your EC2 instances. If you need to assign a new security group to your EC2 instances, select the relevant EC2 instances, then choose “Change Security Group” from the “Actions” dropdown as shown below.
Step 5: Validate Credentials with Atlas Live Migration Service
At this point, you’ve checked that the database user you’re using to perform the live migration has the appropriate roles and permissions. You’ve also granted the Atlas Live Migration Service network access to your source cluster. Now it’s time to bring it all together.
Enter the hostname of your source cluster
In the Atlas Live Migration modal, enter the hostname and port number of your source cluster. This is the public hostname of the EC2 instance associated with your primary, followed by the port number MongoDB is running on.
Enter the database username and password
Next, enter the database user (and password) that you’re using to perform the migration. You checked that this user had the appropriate roles in step 1 of this guide.

SSL Details

If your source cluster requires SSL, you can add the contents of the Certificate Authority file directly into Atlas, allowing the Live Migration Service to connect with the --sslCAFile option.

Next, click “Validate” at the bottom of the Live Migration modal. This will run some quick checks in the background to ensure that the Live Migration Service has all the necessary permissions to perform the migration. If validation passes, move on to the next step. If not, here are a couple of things to double-check:
  • You have whitelisted Atlas on your source cluster.
  • The provided user credentials, if any, exist on the source cluster and have the required roles and permissions.
  • The SSL toggle is enabled only if the source cluster requires it.
  • The CA file provided, if any, is valid and correct.
Step 6: Perform the Live Migration
Click the “Start Migration” button at the bottom of the Live Migration modal.
Once the migration process begins, you will see a progress bar in the cluster card associated with your target cluster (shown below). A countdown timer indicates how much time remains before your target cluster is in sync with your source cluster, at which point you can begin the cutover process.
While you wait, we recommend setting up IP whitelists / VPC Peering for your target cluster in Atlas and setting up database users so you can have your new connection string ready to go. When the countdown timer and the “START CUTOVER” button turn green as shown below, proceed to the next step.
Step 7: Cutover
At this point, your source cluster and new MongoDB Atlas cluster will be in sync. Atlas will maintain this synchronized state for 72 hours (if you need more time, syncing can be extended for 24 hours).
Once you’re ready to begin, click the “START CUTOVER” button and follow the instructions. You will stop writes to your source cluster; this will enable the final writes to your source cluster to be applied to your new Atlas cluster. Once the cluster is caught up, you can end the syncing and restart your application servers pointing to the new cluster using your MongoDB Atlas connection string and database credentials.