MongoDB Atlas With Terraform - Cluster and Backup Policies
Rate this tutorial
In this tutorial, I will show you how to create a MongoDB cluster in Atlas using Terraform. We saw in a previous article how to create an API key to start using Terraform and create our first project module. Now, we will go ahead and create our first cluster. If you don't have an API key and a project, I recommend you look at the previous article.
This article is for anyone who intends to use or already uses infrastructure as code (IaC) on the MongoDB Atlas platform or wants to learn more about it.
Everything we do here is contained in the provider/resource documentation: mongodbatlas_advanced_cluster | Resources | mongodb/mongodbatlas | Terraform
At this point, we will create our first replica set cluster using Terraform in MongoDB Atlas. As discussed in the previous article, Terraform is a powerful infrastructure-as-code tool that allows you to manage and provision IT resources in an efficient and predictable way. By using it in conjunction with MongoDB Atlas, you can automate the creation and management of database resources in the cloud, ensuring a consistent and reliable infrastructure.
Before we begin, make sure that all the prerequisites mentioned in the previous article are properly configured: Install Terraform, create an API key in MongoDB Atlas, and set up a project in Atlas. These steps are essential to ensure the success of creating your replica set cluster.
The first step is to configure the Terraform provider for MongoDB Atlas. This will allow Terraform to communicate with the MongoDB Atlas API and manage resources within your account. Add the following block of code to your provider.tf file:
In the previous article, we configured the Terraform provider by directly entering our public and private keys. Now, in order to adopt more professional practices, we have chosen to use environment variables for authentication. The MongoDB Atlas provider, like many others, supports several authentication methodologies. The safest and most recommended option is to use environment variables. This implies only defining the provider in our Terraform code and exporting the relevant environment variables where Terraform will be executed, whether in the terminal, as a secret in Kubernetes, or a secret in GitHub Actions, among other possible contexts. There are other forms of authentication, such as using MongoDB CLI, AWS Secrets Manager, directly through variables in Terraform, or even specifying the keys in the code. However, to ensure security and avoid exposing our keys in accessible locations, we opt for the safer approaches mentioned.
Inside the versions.tf file, you will start by specifying the version of Terraform that your project requires. This is important to ensure that all users and CI/CD environments use the same version of Terraform, avoiding possible incompatibilities or execution errors. In addition to defining the Terraform version, it is equally important to specify the versions of the providers used in your project. This ensures that resources are managed consistently. For example, to set the MongoDB Atlas provider version, you would add a
required_providers block inside the Terraform block, as shown below:After configuring the version file and establishing the Terraform and provider versions, the next step is to define the cluster resource in MongoDB Atlas. This is done by creating a .tf file, for example main.tf, where you will specify the properties of the desired cluster. As we are going to make a module that will be reusable, we will use variables and default values so that other calls can create clusters with different architectures or sizes, without having to write a new module.
I will look at some attributes and parameters to make this clear.
In this first block, we are specifying the name of our cluster through the name parameter, its type (which can be a
REPLICASET, SHARDED, or GEOSHARDED), and if we have backup and point in time activated, in addition to the database version and the amount of storage for the cluster.Here, we are specifying some advanced settings. Many of these values will not be specified in the .tfvars as they have default values in the variables.tf file.
Parameters include the type of read/write concern, oplog size in MB, TLS protocol, whether JavaScript will be enabled in MongoDB, and transaction lifetime limit in seconds. no_table_scan is for when the cluster disables the execution of any query that requires a collection scan to return results, when true. There are more parameters that you can look at in the documentation, if you have questions.
At this moment, we are placing the number of shards we want, in case our cluster is not a REPLICASET. In addition, we specify the configuration of the cluster, region, cloud, priority for failover, autoscaling, electable, analytics, and read-only node configurations, in addition to its autoscaling configurations.
Next, we create a dynamic block to loop for each tag variable we include. In addition, we specify the BI connector, if desired, and the lifecycle block. Here, we are only specifying
disk_size_gb for an example, but it is recommended to read the documentation that has important warnings about this block, such as including instance_size, as autoscaling can change and you don't want to accidentally retire an instance during peak times.Finally, we create the backup block, which contains the policies and settings regarding the backup of our cluster.
This module, while detailed, encapsulates the full functionality offered by the
mongodbatlas_advanced_cluster and mongodbatlas_cloud_backup_schedule resources, providing a comprehensive approach to creating and managing clusters in MongoDB Atlas. It supports the configuration of replica set, sharded, and geosharded clusters, meeting a variety of scalability and geographic distribution needs.One of the strengths of this module is its flexibility in configuring backup policies, allowing fine adjustments that precisely align with the requirements of each database. This is essential to ensure resilience and effective data recovery in any scenario. Additionally, the module comes with vertical scaling enabled by default, in addition to offering advanced storage auto-scaling capabilities, ensuring that the cluster dynamically adjusts to the data volume and workload.
To complement the robustness of the configuration, the module allows the inclusion of analytical nodes and read-only nodes, expanding the possibilities of using the cluster for scenarios that require in-depth analysis or intensive read operations without impacting overall performance.
The default configuration includes smart preset values, such as the MongoDB version, which is set to "7.0" to take advantage of the latest features while maintaining the option to adjust to specific versions as needed. This “best practices” approach ensures a solid starting point for most projects, reducing the need for manual adjustments and simplifying the deployment process.
Additionally, the ability to deploy clusters in any region and cloud provider — such as AWS, Azure, or GCP — offers unmatched flexibility, allowing teams to choose the best solution based on their cost, performance, and compliance preferences.
In summary, this module not only facilitates the configuration and management of MongoDB Atlas clusters with an extensive range of options and adjustments but also promotes secure and efficient configuration practices, making it a valuable tool for developers and database administrators in implementing scalable and reliable data solutions in the cloud.
The use of the lifecycle directive with the
ignore_changes option in the Terraform code was specifically implemented to accommodate manual upscale situations of the MongoDB Atlas cluster, which should not be automatically reversed by Terraform in subsequent executions. This approach ensures that, after a manual increase in storage capacity (disk_size_gb) or other specific replication configurations (replication_specs), Terraform does not attempt to undo these changes to align the resource state with the original definition in the code. Essentially, it allows configuration adjustments made outside of Terraform, such as an upscale to optimize performance or meet growing demands, to remain intact without being overwritten by future Terraform executions, ensuring operational flexibility while maintaining infrastructure management as code.In the variable.tf file, we create variables with default values:
We configured a file called locals.tf specifically to define two exemplary tags, aiming to identify the name of our application and the environment in which it operates. If you prefer, it is possible to adopt an external tag module, similar to those used in AWS, and integrate it into this configuration.
In this article, we embrace the use of data sources in Terraform to establish a dynamic connection with existing resources, such as our MongoDB Atlas project. Specifically, in the data.tf file, we define a
mongodbatlas_project data source to access information about an existing project based on its name:Here,
var.project_name refers to the name of the project we want to query, an approach that allows us to keep our configuration flexible and reusable. The value of this variable can be provided in several ways, significantly expanding the possibilities for using our infrastructure as code.The terraform.tfvars file is used to define variable values that will be applied in the Terraform configuration, making infrastructure as code more dynamic and adaptable to the specific needs of each project or environment. In your case, the terraform.tfvars file contains essential values for creating a cluster in MongoDB Atlas, including details such as the project name, cluster characteristics, and auto-scaling settings. See below how these definitions apply:
These values defined in terraform.tfvars are used by Terraform to populate corresponding variables in your configuration. For example, if you have a module or feature that creates a cluster in MongoDB Atlas, you can reference these variables directly to configure properties such as the project name, cluster settings, and regional specifications. This allows significant flexibility in customizing your infrastructure based on different environments or project requirements.
The file structure was as follows:
- main.tf: In this file, we will define the main resource, the mongodbatlas_advanced_cluster, and mongodbatlas_cloud_backup_schedule. Here, you have configured the cluster and backup routines.
- provider.tf: This file is where we define the provider we are using — in our case, mongodbatlas. We will specify using environment variables, as mentioned previously.
- terraform.tfvars: This file contains the variables that will be used in our cluster. For example, the cluster name, cluster information, version, size, among others.
- variable.tf: Here, we define the variables mentioned in the terraform.tfvars file, specifying the type and optionally a default value.
- version.tf: This file is used to specify the version of Terraform and the providers we are using.
- data.tf: Here, we specify a data source that will bring us information about our created project. We will search for its name and for our module, it will give us the project ID.
- locals.tf: We specify example tags to use in our cluster.
Now is the time to apply. =D
We run a
terraform init in the terminal in the folder where the files are located so that it downloads the providers, modules, etc…Note: Remember to export the environment variables with the public and private keys.
Now, we run
terraform init.Now that init has worked, let's run
terraform plan and evaluate what will happen:Show! It was exactly the output we expected to see, the creation of a cluster resource with the backup policies. Let's apply this!
When running the
terraform apply command, you will be prompted for approval with yes or no. Type yes.This process took eight minutes and 40 seconds to execute. I shortened the log output, but don't worry if this step takes time.
Now, let’s look in Atlas to see if the cluster was created successfully…
We were able to create our first replica set with a standard backup policy with PITR and scheduled snapshots.
In this tutorial, we saw how to create the first cluster in our project created in the last article. We created a module that also includes a backup policy. In an upcoming article, we will look at how to create an API key and user using Terraform and Atlas.
To learn more about MongoDB and various tools, I invite you to visit the Developer Center to read the other articles.
Rate this tutorial