Atlas Cluster Automation Using Scheduled Triggers

Brian Leonard

#MongoDB Atlas

Every action you can take in the Atlas user interface is backed by a corresponding REST API, which allows you to easily bring automation to your Atlas deployments. Some of the more common forms of Atlas automation occur on a schedule, such as pausing a cluster that’s only used for testing in the evenings and resuming the cluster again in the morning.

Having an API to automate Atlas actions is great, but you’re still on the hook for writing the script that calls the API, finding a place to host the script, and setting up the job to call the script on your desired schedule. This is where Atlas Scheduled Triggers come to the rescue.

In this article, I will show you how a Scheduled Trigger can be used to easily incorporate automation into your environment. In addition to pausing and unpausing a cluster, I’ll similarly show how cluster scale up and down events could also be placed on a schedule. Both of these activities allow you to save costs for when you either don’t need the cluster (paused), or don’t need it to support peak workloads (scale down).

Preparation

Identify Trigger Host Cluster

Atlas triggers do not fire when their associated cluster is paused, so the trigger needs to be linked to a cluster that never pauses. Fortunately, Atlas supports a forever free sandbox cluster (M0). If you don’t have a cluster you can use that will never be paused (such as a production cluster), create an M0 cluster to use for this exercise.

Note the Project ID

We’ll need to pass the project ID as a path parameter in our API calls. You can find it under the Settings menu on the left:

MongoDB Atlas Settings Menu

Project ID

Generate an API Key

Even if you already have an API Key, it’s good to generate a new one for this use case so we can easily identify the API was called by the Atlas Trigger.

At the Organization level (not the Project level), select Access from the menu on the left:

Organization Menu

Then select the API Keys tab.

Create a new key, giving it a good description. Assign the key Organization Owner permissions, which is the minimum required to execute Stitch functions.

API Key Information

Click Next and make a note of your Private Key:

Private Key Information

Don’t worry about the whitelist entry. By leaving the whitelist empty, we’ll be able to call the API from our trigger.

Scheduled Pause

The ability to pause and resume a cluster is supported by the Modify Cluster API. To begin, select Triggers from the menu on the left:

Services Menu

And add a trigger.

Set the Trigger Type to Scheduled and the name to pauseClusters:

Add Trigger

As for the schedule, you have the full power of CRON Expressions at your fingertips. For this exercise, let’s assume we want to pause the cluster every evening at 6 pm. Select Advanced and set the CRON schedule to 0 18 * * *:

Schedule Type

You can use the Next Events to validate the job will run when you desire.

The next step is to link trigger to the cluster, and it’s important to link to the cluster we identified above that will always be running. I created a M0 cluster called Automation for this purpose:

Link Cluster

Finally, we need to provide the function that will run when the trigger fires. Replace the provided function code with the following, filling in your values for username, password, projectID and clusterNames. Note the function as written can pause multiple clusters, providing the cluster names as an array:

exports = async function() {
  
  const username = '<API Public Key>';
  const password = '<API Private Key>';
  const projectID = '<Project ID>';
  const clusterNames = ['<Cluster Name>', '<Cluster Name>'];
  const body = {paused: true};
  var n = 0;
  var result = "";
  
  for (n in clusterNames) {
    result = await context.functions.execute('modifyCluster', username, password, projectID, clusterNames[n], body)
    console.log(EJSON.stringify(result));
    
    if (result.error) {
      return result;
    }
  }

  return clusterNames.length + " clusters paused"; 
};

And Save the trigger.

If you attempt to run the trigger it will fail because it’s calling a wrapper function, modifyCluster. Let’s implement that now...

Implement Modify Cluster

I’m only demonstrating a couple of things you can do with cluster automation, but the sky is really limitless. The following modifyCluster function is a generic wrapper around the Modify Cluster API for calling the API from Stitch (or Node.js for that matter).

When we created the Atlas Trigger, behind the scenes, a Stitch application was created to run the trigger. It’s in that application where we’ll host our modifyCluster function.

Select Stitch from the menu on the left:

Services Menu

And you’ll find a Triggers_StitchApp has been created for us:

Triggers StitchApp

Open the app and select Functions from the menu on the left:

Build Menu

Click the Create New Function button. Name the function modifyCluster:

Function Settings

Leave the other fields at their defaults and click Save to open the Function Editor. Paste the following code into the editor:

/*
 * Modifies the cluster as defined by the body parameter. 
 * See https://docs.atlas.mongodb.com/reference/api/clusters-modify-one/
 *
 */
exports = async function(username, password, projectID, clusterName, body) {
  
  const arg = { 
    scheme: 'https', 
    host: 'cloud.mongodb.com', 
    path: 'api/atlas/v1.0/groups/' + projectID + '/clusters/' + clusterName, 
    username: username, 
    password: password,
    headers: {'Content-Type': ['application/json'], 'Accept-Encoding': ['bzip, deflate']}, 
    digestAuth:true,
    body: JSON.stringify(body)
  };
  
  // The response body is a BSON.Binary object. Parse it and return.
  response = await context.http.patch(arg);

  return EJSON.parse(response.body.text()); 
};

As stated above, this is simply a generic wrapper around the modify cluster API. Save the function then click REVIEW & DEPLOY CHANGES:

Review & Deploy Changes

Test the Trigger

Click the < Back to Atlas link in the upper left-hand corner of the Stitch application and open your pauseClusters trigger and click Run to test it. You should see output similar to the following:

Run Output

More importantly, you should see your cluster entering a paused state:

Paused Cluster

Resume the Cluster

You could opt to manually resume the cluster(s) as it’s needed. But for completeness, let’s assume we want the cluster(s) to automatically resume at 8 am every weekday morning.

Add a new Atlas scheduled trigger named resumeClusters. Set the CRON schedule to: 0 8 * * 1-5. The Next Events validates for us this is exactly what we want:

Schedule Type

The function code is almost identical to the pauseCluster. We simply set paused to false and update our return statement to indicate clusters were resumed:

exports = async function() {
  
  const username = '<API Public Key>';
  const password = '<API Private Key>';
  const projectID = '<Project ID>';
  const clusterNames = ['<Cluster Name>', '<Cluster Name>'];
  const body = {paused: false};
  var n = 0;
  var result = "";
  
  for (n in clusterNames) {
    result = await context.functions.execute('modifyCluster', username, password, projectID, clusterNames[n], body)
    console.log(EJSON.stringify(result));
    
    if (result.error) {
      return result;
    }
  }

  return clusterNames.length + " clusters resumed"; 
};

Scaling Up and Down

It’s not uncommon to have workloads that are more demanding during certain hours of the day or days of the week. Rather than running your cluster to support peak capacity, you can use this same approach to schedule your cluster to scale up and down as your workload requires it.

Note, Atlas Clusters already support Auto-Scaling, which may very well suit your needs. The approach described here will let you definitively control when your cluster scales up and down.

Let’s say we want to scale up our cluster every day at 9 am before our store opens for business.

Add a new Atlas scheduled trigger named scaleClusterUp. Set the CRON schedule to: 0 9 * * *.

Here’s the function code. It’s very similar to before, except the body’s been changed to alter the provider settings:

exports = async function() {
  
  const username = '<API Public Key>';
  const password = '<API Private Key>';
  const projectID = '<Project ID>';
  const clusterNames = '<Cluster Name>';
  const body =    {
      "providerSettings" : {
        "providerName" : "AWS",
        "instanceSizeName" : "M20"
      }
    };
  
  result = await context.functions.execute('modifyCluster', username, password, projectID, clusterName, body);
  console.log(EJSON.stringify(result));
  
  if (result.error) {
    return result;
  }

  return clusterName + " scaled up"; 
};

Scaling a cluster back down would simply be another trigger, scheduled to run when you want, using the same code above, setting the instanceSizeName to whatever you desire.

And that’s it. I hope you find this beneficial. You should be able to use the techniques described here to easily call any MongoDB Atlas API from Stitch.