EventGet 50% off your ticket to MongoDB.local NYC on May 2. Use code Web50!Learn more >>
MongoDB Developer
Atlas
plus
Sign in to follow topics
MongoDB Developer Centerchevron-right
Developer Topicschevron-right
Productschevron-right
Atlaschevron-right

Atlas Cluster Automation Using Scheduled Triggers

Brian Leonard11 min read • Published Jan 08, 2020 • Updated Jan 02, 2024
AtlasJavaScript
Facebook Icontwitter iconlinkedin icon
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty

Atlas Cluster Automation Using Scheduled Triggers

Every action you can take in the Atlas user interface is backed by a corresponding Administration API, which allows you to easily bring automation to your Atlas deployments. Some of the more common forms of Atlas automation occur on a schedule, such as pausing a cluster that’s only used for testing in the evening and resuming the cluster again in the morning.
Having an API to automate Atlas actions is great, but you’re still on the hook for writing the script that calls the API, finding a place to host the script, and setting up the job to call the script on your desired schedule. This is where Atlas Scheduled Triggers come to the rescue.
In this article I will show you how a Scheduled Trigger can be used to easily incorporate automation into your environment. In addition to pausing and unpausing a cluster, I’ll similarly show how cluster scale up and down events could also be placed on a schedule. Both of these activities allow you to save on costs for when you either don’t need the cluster (paused), or don’t need it to support peak workloads (scale down).

Architecture

Three example scheduled triggers are provided in this solution. Each trigger has an associated trigger function. The bulk of the work is handled by the modifyCluster function, which as the name implies is a generic function for making modifications to a cluster. It's a wrapper around the Atlas Update Configuration of One Cluster Admin API.
Architecture

Preparation

Generate an API Key

In order to call the Atlas Administrative APIs, you'll first need an API Key with the Organization Owner role. API Keys are created in the Access Manager. At the Organization level (not the Project level), select Access Manager from the menu on the left:
Access Manager
Then select the API Keys tab.
Create a new key, giving it a good description. Assign the key Organization Owner permissions, which will allow it to manage any of the projects in the organization.
API Key
Click Next and make a note of your Private Key:
Save API Key
Let's limit who can use our API key by adding an access list. In our case, the API key is going to be used by a Trigger which is a component of Atlas App Services. You will find the list of IP addresses used by App Services in the documentation under Firewall Configuration. Note, each IP address must be added individually. Here's an idea you can vote for to get this addressed: Ability to provide IP addresses as a list for Network Access
Add Access List Entry
API Access List
Click Done.

Deployment

Create a Project for Automation

Since this solution works across your entire Atlas organization, I like to host it in its own dedicated Atlas Project.
Create a Project

Create an Application

We will host our trigger in an Atlas App Services Application. To begin, just click the App Services tab:
App Services
You'll see that App Services offers a bunch of templates to get you started. For this use case, just select the first option to Build your own App:
Welcome to App Services
You'll then be presented with options to link a data source, name your application and choose a deployment model. The current iteration of this utility doesn't use a data source, so you can ignore that step (a free cluster for you regardless). You can also leave the deployment model at its default (Global), unless you want to limit the application to a specific region.
I've named the application Automation App:
Welcome to App Services
Click Create App Service. If you're presented with a set of guides, click Close Guides as today I am your guide.
From here, you have the option to simply import the App Services application and adjust any of the functions to fit your needs. If you prefer to build the application from scratch, skip to the next section.

Import Option

Step 1: Store the API Secret Key

The extract has a dependency on the API Secret Key, thus the import will fail if it is not configured beforehand.
Use the Values menu on the left to Create a Secret named AtlasPrivateKeySecret containing your private key (the secret is not in quotes):
Create Secret

Step 2: Install the App Services CLI

The App Services CLI is available on npm. To install the App Services CLI on your system, ensure that you have Node.js installed and then run the following command in your shell:

Step 3: Extract the Application Archive

Download and extract the AutomationApp.zip.

Step 4: Log into Atlas

To configure your app with App Services CLI, you must log in to Atlas using your API keys:

Step 5: Get the Application ID

Select the App Settings menu and copy your Application ID:
App ID

Step 6: Import the Application

Run the following appservices push command from the directory where you extracted the export:
After the import, replace the AtlasPublicKey with your API public key value.
Atlas Public Key

Review the Imported Application

The imported application includes 3 self-explanatory sample scheduled triggers:
Triggers
The 3 triggers have 3 associated Functions. The pauseClustersTrigger and resumeClustersTrigger function supply a set of projects and clusters to pause, so these need to be adjusted to fit your needs:
All 3 trigger functions call the modifyCluster function, where the bulk of the work is done.
In addition, you'll find two utility functions, getProjectClusters and getProjects. These functions are not utilized in this solution, but are provided for reference if you wanted to further automate these processes (that is, removing the hard coded project IDs and cluster names in the trigger functions):
Functions
Now that you have reviewed the draft, as a final step go ahead and deploy the App Services application.
Review Draft & Deploy

Build it Yourself Option

To understand what's included in the application, here are the steps to build it yourself from scratch.

Step 1: Store the API Keys

The functions we need to create will call the Atlas Administration APIs, so we need to store our API Public and Private Keys, which we will do using Values & Secrets. The sample code I provide references these values as AtlasPublicKey and AtlasPrivateKey, so use those same names unless you want to change the code where they’re referenced.
You'll find Values under the BUILD menu:
Values
First, create a Value for your public key (note, the key is in quotes):
Atlas Public Key
Create a Secret containing your private key (the secret is not in quotes):
Create Secret
The Secret cannot be accessed directly, so create a second Value that links to the secret:
Link to Secret

Step 2: Note the Project ID(s)

We need to note the IDs of the projects that have clusters we want to automate. Click the 3 dots in the upper left corner of the UI to open the Project Settings:
Project Settings
Under which you’ll find your Project ID:
Project ID

Step 3: Create the Functions

I will create two functions, a generic function to modify a cluster and a trigger function to iterate over the clusters to be paused.
You'll find Functions under the BUILD menu:
Functions

modifyCluster

I’m only demonstrating a couple of things you can do with cluster automation, but the sky is really limitless. The following modifyCluster function is a generic wrapper around the Modify One Multi-Cloud Cluster from One Project API for calling the API from App Services (or Node.js for that matter).
Create a New Function named modifyCluster. Set the function to Private as it will only be called by our trigger. The other default settings are fine:
Modify Cluster Function
Switch to the Function Editor tab and paste the following code:
To test this function, you need to supply an API key, an API secret, a project Id, an associated cluster name to modify, and a payload containing the modifications you'd like to make. In our case it's simply setting the paused property.
Note: By default, the Console supplies 'Hello world!' when test running a function, so my function code tests for that input and provides some default values for easy testing.
Console
Press the Run button to see the results, which will appear in the Result window:
Run
And you should find you cluster being resumed (or paused):
Cluster

pauseClustersTrigger

This function will be called by a trigger. As it's not possible to pass parameters to a scheduled trigger, it uses a hard-coded list of project Ids and associated cluster names to pause. Ideally these values would be stored in a collection with a nice UI to manage all of this, but that's a job for another day :-).
In the appendix of this article, I provide functions that will get all projects and clusters in the organization. That would create a truly dynamic operation that would pause all clusters. You could then alternatively refactor the code to use an exclude list instead of an allow list.

Step 4: Create Trigger - pauseClusters

The ability to pause and resume a cluster is supported by the Update Configuration of One Cluster API. To begin, select Triggers from the menu on the left:
Triggers Menu
And add a Trigger.
Set the Trigger Type to Scheduled and the name to pauseClusters:
Add Trigger
As for the schedule, you have the full power of CRON Expressions at your fingertips. For this exercise, let’s assume we want to pause the cluster every evening at 6pm. Select Advanced and set the CRON schedule to 0 22 * * *.
Note, the time is in GMT, so adjust accordingly for your timezone. As this cluster is running in US East, I’m going to add 4 hours:
Schedule Type
Check the Next Events window to validate the job will run when you desire.
The final step is to select the function for the trigger to execute. Select the pauseClustersTrigger function.
Trigger Function
And Save the trigger.
The final step is to REVIEW DRAFT & DEPLOY.
Review Draft & Deploy

Resume the Cluster

You could opt to manually resume the cluster(s) as it’s needed. But for completeness, let’s assume we want the cluster(s) to automatically resume at 8am US East every weekday morning.
Duplicate the pauseClustersTrigger function to a new function named resumeClustersTriggger
Duplicate Function
At a minimum, edit the function code setting paused to false. You could also adjust the projectIDs and clusterNames to a subset of projects to resume:
Then add a new scheduled trigger named resumeClusters. Set the CRON schedule to: 0 12 * * 1-5. The Next Events validates for us this is exactly what we want:
Schedule Type Resume

Create Trigger: Scaling Up and Down

It’s not uncommon to have workloads that are more demanding during certain hours of the day or days of the week. Rather than running your cluster to support peak capacity, you can use this same approach to schedule your cluster to scale up and down as your workload requires it.
NOTE: Atlas Clusters already support Auto-Scaling, which may very well suit your needs. The approach described here will let you definitively control when your cluster scales up and down.
Let’s say we want to scale up our cluster every day at 9am before our store opens for business.
Add a new function named scaleClusterUpTrigger. Here’s the function code. It’s very similar to before, except the body’s been changed to alter the provider settings:
Then add a scheduled trigger named scaleClusterUp. Set the CRON schedule to: 0 13 * * *.
Scaling a cluster back down would simply be another trigger, scheduled to run when you want, using the same code above, setting the instanceSizeName to whatever you desire.
And that’s it. I hope you find this beneficial. You should be able to use the techniques described here to easily call any MongoDB Atlas Admin API endpoint from Atlas App Services.

Appendix

getProjects

This standalone function can be test run from the App Services console to see the list of all the projects in your organization. You could also call it from other functions to get a list of projects:

getProjectClusters

Another example function that will return the cluster details for a provided project.
Note, to test this function, you need to supply a projectId. By default, the Console supplies ‘Hello world!’, so I test for that input and provide some default values for easy testing.

Facebook Icontwitter iconlinkedin icon
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty
Related
Article

Listen Along at Scale Up with Atlas Application Services


Jun 23, 2022 | 3 min read
Tutorial

Boosting AI: Build Your Chatbot Over Your Data With MongoDB Atlas Vector Search and LangChain Templates Using the RAG Pattern


Jan 16, 2024 | 7 min read
Tutorial

Enhancing LLM Accuracy Using MongoDB Vector Search and Unstructured.io Metadata


Dec 04, 2023 | 12 min read
Tutorial

Building a Restaurant Locator Using Atlas, Neurelo, and AWS Lambda


Mar 08, 2024 | 8 min read
Table of Contents