Anaiya Raisinghani

Nic Raboy

In the developer community, ensuring your projects run accurately regardless of the environment can be a pain. Whether it’s trying to recreate a demo from an online tutorial or working on a code review, hearing the words, “Well, it works on my machine…” can be frustrating. Instead of spending hours debugging, we want to introduce you to a platform that will change your developer experience: [Docker](https://www.docker.com/). 

Docker is a great tool to learn because it provides developers with the ability for their applications to be used easily between environments, and it's resource-efficient in comparison to virtual machines. This tutorial will gently guide you through how to navigate Docker, along with how to integrate [Go](https://golang.org/) on the platform. We will be using this project to connect to our previously built MongoDB Atlas Search Cluster made for using [Synonyms in Atlas Search](https://www.mongodb.com/developer/how-to/get-hyped-synonyms-atlas-search/). Stay tuned for a fun read on how to learn all the above while also expanding your Gen-Z slang knowledge from our synonyms cluster. Get hyped! 

## The Prerequisites

There are a few requirements that must be met to be successful with this tutorial.

- A M0 or better MongoDB Atlas cluster
- Docker Desktop

To use MongoDB with the Golang driver, you only need a free M0 cluster. To create this cluster, follow the instructions listed on the [MongoDB documentation](https://docs.atlas.mongodb.com/tutorial/create-new-cluster/). However, we’ll be making many references to a [previous tutorial](https://www.mongodb.com/developer/how-to/get-hyped-synonyms-atlas-search/) where we used Atlas Search with custom synonyms.

Since this is a Docker tutorial, you’ll need [Docker Desktop](https://www.docker.com/products/docker-desktop). You don’t actually need to have Golang configured on your host machine because Docker can take care of this for us as we progress through the tutorial.

## Building a Go API with the MongoDB Golang Driver 

Like previously mentioned, you don’t need Go installed and configured on your host computer to be successful. However, it wouldn’t hurt to have it in case you wanted to test things prior to creating a Docker image.

On your computer, create a new project directory, and within that project directory, create a **src** directory with the following files:

- go.mod
- main.go

The **go.mod** file is our dependency management file for Go modules. It could easily be created manually or by using the following command:

```bash
go mod init
```

The **main.go** file is where we’ll keep all of our project code.

Starting with the **go.mod** file, add the following lines:

```
module github.com/mongodb-developer/docker-golang-example
go 1.15
require go.mongodb.org/mongo-driver v1.7.0
require github.com/gorilla/mux v1.8.0
```

Essentially, we’re defining what version of Go to use and the modules that we want to use. For this project, we’ll be using the MongoDB Go driver as well as the [Gorilla Web Toolkit](https://www.gorillatoolkit.org/).

This brings us into the building of our simple API.

Within the **main.go** file, add the following code:

```golang
package main

import (
	"context"
	"encoding/json"
	"fmt"
	"net/http"
	"os"
	"time"

	"github.com/gorilla/mux"
	"go.mongodb.org/mongo-driver/bson"
	"go.mongodb.org/mongo-driver/mongo"
	"go.mongodb.org/mongo-driver/mongo/options"
)

var client *mongo.Client
var collection *mongo.Collection

type Tweet struct {
	ID       int64  `json:"_id,omitempty" bson:"_id,omitempty"`
	FullText string `json:"full_text,omitempty" bson:"full_text,omitempty"`
	User     struct {
		ScreenName string `json:"screen_name" bson:"screen_name"`
	} `json:"user,omitempty" bson:"user,omitempty"`
}

func GetTweetsEndpoint(response http.ResponseWriter, request *http.Request) {}
func SearchTweetsEndpoint(response http.ResponseWriter, request *http.Request) {}

func main() {
	fmt.Println("Starting the application...")
	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
	defer cancel()
	client, err := mongo.Connect(ctx, options.Client().ApplyURI(os.Getenv("MONGODB_URI")))
	defer func() {
		if err = client.Disconnect(ctx); err != nil {
			panic(err)
		}
	}()
collection = client.Database("synonyms").Collection("tweets")
	router := mux.NewRouter()
	router.HandleFunc("/tweets", GetTweetsEndpoint).Methods("GET")
	router.HandleFunc("/search", SearchTweetsEndpoint).Methods("GET")
	http.ListenAndServe(":12345", router)
}
```

There’s more to the code, but before we see the rest, let’s start breaking down what we have above to make sense of it.

You’ll probably notice our `Tweets` data structure:

```golang
type Tweet struct {
	ID       int64  `json:"_id,omitempty" bson:"_id,omitempty"`
	FullText string `json:"full_text,omitempty" bson:"full_text,omitempty"`
	User     struct {
		ScreenName string `json:"screen_name" bson:"screen_name"`
	} `json:"user,omitempty" bson:"user,omitempty"`
}
```

Earlier in the tutorial, we mentioned that this example is heavily influenced by a [previous tutorial](https://www.mongodb.com/developer/how-to/get-hyped-synonyms-atlas-search/) that used Twitter data. We highly recommend you take a look at it. This data structure has some of the fields that represent a tweet that we scraped from Twitter. We didn’t map all the fields because it just wasn’t necessary for this example.

Next, you’ll notice the following:

```golang
func GetTweetsEndpoint(response http.ResponseWriter, request *http.Request) {}
func SearchTweetsEndpoint(response http.ResponseWriter, request *http.Request) {}
```

These will be the functions that hold our API endpoint logic. We’re going to skip these for now and focus on understanding the connection and configuration logic.

As of now, most of what we’re interested in is happening in the `main` function.

The first thing we’re doing is connecting to MongoDB:

```golang
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
client, err := mongo.Connect(ctx, options.Client().ApplyURI(os.Getenv("MONGODB_URI")))
defer func() {
	if err = client.Disconnect(ctx); err != nil {
		panic(err)
	}
}()
collection = client.Database("synonyms").Collection("tweets")
```

You’ll probably notice the `MONGODB_URI` environment variable in the above code. It’s not a good idea to hard-code the MongoDB connection string in the application. This prevents us from being flexible and it could be a security risk. Instead, we’re using environment variables that we’ll pass in with Docker when we deploy our containers.

You can visit the MongoDB Atlas dashboard for your URI string.

The database we plan to use is `synonyms` and we plan to use the `tweets` collection, both of which we talked about in that previous tutorial.

After connecting to MongoDB, we focus on configuring the Gorilla Web Toolkit:

```golang
router := mux.NewRouter()
router.HandleFunc("/tweets", GetTweetsEndpoint).Methods("GET")
router.HandleFunc("/search", SearchTweetsEndpoint).Methods("GET")
http.ListenAndServe(":12345", router)
```

In this code, we are defining which endpoint path should route to which function. The functions are defined, but we haven’t yet added any logic to them. The application itself will be serving on port 12345.

As of now, the application has the necessary basic connection and configuration information. Let’s circle back to each of those endpoint functions.

We’ll start with the `GetTweetsEndpoint` because it will work fine with an M0 cluster:

```golang
func GetTweetsEndpoint(response http.ResponseWriter, request *http.Request) {
	response.Header().Set("content-type", "application/json")
	var tweets []Tweet
	ctx, _ := context.WithTimeout(context.Background(), 30*time.Second)
	cursor, err := collection.Find(ctx, bson.M{})
	if err != nil {
		response.WriteHeader(http.StatusInternalServerError)
		response.Write([]byte(`{ "message": "` + err.Error() + `" }`))
		return
	}
	if err = cursor.All(ctx, &tweets); err != nil {
		response.WriteHeader(http.StatusInternalServerError)
		response.Write([]byte(`{ "message": "` + err.Error() + `" }`))
		return
	}
	json.NewEncoder(response).Encode(tweets)
}
```

In the above code, we’re saying that we want to use the `Find` operation on our collection for all documents in that collection, hence the empty filter object.

If there were no errors, we can get all the results from our cursor, load them into a `Tweet` slice, and then JSON encode that slice for sending to the client. The client will receive JSON data as a result.

Now we can look at the more interesting endpoint function.

```golang
func SearchTweetsEndpoint(response http.ResponseWriter, request *http.Request) {
	response.Header().Set("content-type", "application/json")
	queryParams := request.URL.Query()
	var tweets []Tweet
	ctx, _ := context.WithTimeout(context.Background(), 30*time.Second)
	searchStage := bson.D{
		{"$search", bson.D{
			{"index", "synsearch"},
			{"text", bson.D{
				{"query", queryParams.Get("q")},
				{"path", "full_text"},
				{"synonyms", "slang"},
			}},
		}},
	}
	cursor, err := collection.Aggregate(ctx, mongo.Pipeline{searchStage})
	if err != nil {
		response.WriteHeader(http.StatusInternalServerError)
		response.Write([]byte(`{ "message": "` + err.Error() + `" }`))
		return
	}
	if err = cursor.All(ctx, &tweets); err != nil {
		response.WriteHeader(http.StatusInternalServerError)
		response.Write([]byte(`{ "message": "` + err.Error() + `" }`))
		return
	}
	json.NewEncoder(response).Encode(tweets)
}
```

The idea behind the above function is that we want to use an aggregation pipeline for Atlas Search. It does use the synonym information that we outlined in the [previous tutorial](https://www.mongodb.com/developer/how-to/get-hyped-synonyms-atlas-search/).

The first important thing in the above code to note is the following:

```golang
​​queryParams := request.URL.Query()
```

We’re obtaining the query parameters passed with the HTTP request. We’re expecting a `q` parameter to exist with the search query to be used.

To keep things simple, we make use of a single stage for the MongoDB aggregation pipeline:

```golang
searchStage := bson.D{
	{"$search", bson.D{
		{"index", "synsearch"},
		{"text", bson.D{
			{"query", queryParams.Get("q")},
			{"path", "full_text"},
			{"synonyms", "slang"},
		}},
	}},
}
```

In this stage, we are doing a text search with a specific index and a specific set of synonyms. The query that we use for our text search comes from the query parameter of our HTTP request.

Assuming that everything went well, we can load all the results from the cursor into a `Tweet` slice, JSON encode it, and return it to the client that requested it.

If you have Go installed and configured on your computer, go ahead and try to run this application. Just don’t forget to add the `MONGODB_URI` to your environment variables prior.

If you want to learn more about API development with the Gorilla Web Toolkit and MongoDB, check out [this tutorial](https://www.thepolyglotdeveloper.com/2019/02/developing-restful-api-golang-mongodb-nosql-database/) on the subject.

## Configuring a Docker Image for Go with MongoDB

Let’s get started with Docker! If it’s a platform you’ve never used before, it might seem a bit daunting at first, but let us guide you through it, step by step. We will be showing you how to download Docker and get started with setting up your first Dockerfile to connect to our Gen-Z Synonyms Atlas Cluster. 

First things first. Let’s download Docker. This can be done through their [website](https://www.docker.com/products/docker-desktop) in just a couple of minutes. 

Once you have that up and running, it’s time to create your very first Dockerfile. 

At the root of your project folder, create a new **Dockerfile** file with the following content:

```
#get a base image
FROM golang:1.16-buster

MAINTAINER anaiya raisinghani <anaiya.raisinghani@mongodb.com>

WORKDIR /go/src/app
COPY ./src .

RUN go get -d -v
RUN go build -v

CMD ["./docker-golang-example"]
```

This format is what many Dockerfiles are composed of, and a lot of it is heavily customizable and can be edited to fit your project's needs. 

The first step is to grab a base image that you’re going to use to build your new image. You can think of using Dockerfiles as layers to a cake. There are a multitude of different base images out there, or you can use `FROM scratch` to start from an entirely blank image. Since this project is using the programming language Go, we chose to start from the `golang` base image and add the tag `1.16` to represent the version of Go that we plan to use. Whenever you include a tag next to your base image, be sure to set it up with a colon in between, just like this: `golang:1.16`. To learn more about which tag will benefit your project the best, check out [Docker’s documentation](https://hub.docker.com/_/golang?tab=tags&page=1&ordering=last_updated) on the subject.

This site holds a lot of different tags that can be used on a Golang base image. Tags are important because they hold very valuable information about the base image you’re using such as software versions, operating system flavor, etc. 

Let’s run through the rest of what will happen in this Dockerfile!

It's optional to include a `MAINTAINER` for your image, but it’s good practice so that people viewing your Dockerfile can know who created it. It's not necessary, but it’s helpful to include your full name and your email address in the file. 

The `WORKDIR /go/src/app` command is crucial to include in your Dockerfile since `WORKDIR` specifies which working directory you’re in. All the commands after will be run through whichever directory you choose, so be sure to be aware of which directory you’re currently in.

The `COPY ./src .` command allows you to copy whichever files you want from the specified location on the host machine into the Docker image. 

Now, we can use the `RUN` command to set up exactly what we want to happen at image build time before deploying as a container. The first command we have is `RUN go get -d -v`, which will download all of the Go dependencies listed in the **go.mod** file that was copied into the image.. 

Our second `RUN` command is `RUN go build -v`, which will build our project into an executable binary file. 

The last step of this Dockerfile is to use a `CMD` command, `CMD [“./docker-golang-example”]`. This command will define what is run when the container is deployed rather than when the image is built. Essentially we’re saying that we want the built Go application to be run when the container is deployed.

Once you have this Dockerfile set up, you can build and execute your project using your entire MongoDB URI link:

To build the Docker image and deploy the container, execute the following from the command line:

```bash
docker build -t docker-syn-image .
docker run -d -p 12345:12345 -e “MONGODB_URI=YOUR_URI_HERE” docker-syn-image
```

Following these instructions will allow you to run the project and access it from http://localhost:12345. **But**! It’s so tedious. What if we told you there was an easier way to run your application without having to write in the entire URI link? There is! All it takes is one extra step: setting up a Docker Compose file. 

## Setting Up a Docker Compose File to Streamline Deployments

A Docker Compose file is a nice little step to run all your container files and dependencies through a simple command: `docker compose up`.

In order to set up this file, you need to establish a YAML configuration file first. Do this by creating a new file in the root of your project folder, naming it **docker-compose**, and adding **.yml** at the end. You can name it something else if you like, but this is the easiest since when running the `docker compose up` command, you won’t need to specify a file name. Once that is in your project folder, follow the steps below.

This is what your Docker Compose file will look like once you have it all set up: 

```yaml
version: "3.9" 
services:
   web:
       build: .
       ports:
           - "12345:12345"
       environment:
           MONGODB_URI: your_URI_here
```

Let’s run through it!

First things first. Determine which schema version you want to be running. You should be using the most recent version, and you can find this out through [Docker’s documentation](https://docs.docker.com/compose/compose-file/).

Next, define which services, otherwise known as containers, you want to be running in your project. We have included `web` since we are attaching to our Atlas Search cluster. The name isn’t important and it acts more as an identifier for that particular service. Next, specify that you are building your application, and put in your `ports` information in the correct spot. For the next step, we can set up our `environment` as our MongoDB URI and we’re done! 

Now, run the command `docker compose up` and watch the magic happen. Your container should build, then run, and you’ll be able to connect to your port and see all the tweets!

## Conclusion

This tutorial has now left you equipped with the knowledge you need to build a Go API with the MongoDB Golang driver, create a Dockerfile, create a Docker Compose file, and connect your newly built container to a MongoDB Atlas Cluster. 

Using these new platforms will allow you to take your projects to a whole new level. 

If you’d like to take a look at the code used in our project, you can access it on [GitHub](https://github.com/mongodb-developer/docker-golang-example).

Using Docker or Go, but have a question? Check out the [MongoDB Community Forums](https://community.mongodb.com)!

Learn how to create and deploy Golang-powered micro-services that interact with MongoDB using Docker.

Get Hyped: Using Docker + Go with MongoDB

Pierre Petersson

Johannes Brännström

## Why CI/CD?

To increase the speed and quality of development, you may use continuous delivery strategies to manage and deploy your application code changes. However, continuous delivery for databases is often a manual process.

Adopting continuous integration and continuous delivery (CI/CD) for managing the lifecycle of a database has the following benefits:

* An automated multi-environment setup enables you to move faster and focus on what really matters.
* The confidence level of the changes applied increases.
* The process is easier to reproduce.
* All changes to database configuration will be traceable.

### Why CI/CD for MongoDB Atlas?

[MongoDB Atlas](https://www.mongodb.com/cloud/atlas) is a multi-cloud application data platform, providing an integrated suite of cloud database and data services to accelerate and simplify how you build with data. MongoDB Atlas also provides a comprehensive [API](https://docs.atlas.mongodb.com/api/atlas-admin-api/), making CI/CD for the actual data platform itself possible.

In this blog, we’ll demonstrate how to set up CI/CD for MongoDB Atlas, in a typical production setting. The intended audience is developers, solutions architects, and database administrators with knowledge of MongoDB Atlas, AWS, and Terraform.

## Our CI/CD Solution Requirements

* Ensure that each environment (dev, test, prod) is isolated to minimize blast radius in case of a human error or from a security perspective. MongoDB Atlas Projects and API Keys will be utilized to enable environment isolation.
* All services used in this solution will use managed services. This to minimize the time needed to spend on managing infrastructure.
* Minimize commercial agreements required. Use as much as possible from AWS and the Atlas ecosystem so that there is no need to purchase external tooling, such as HashiCorp Vault. 
* Minimize time spent on installing local dev tooling, such as git and Terraform. The solution will provide a docker image, with all tooling required to run provisioning of Terraform templates. The same image will be used to also run the pipeline in AWS CodeBuild. 

## Implementation

Enough talk—let’s get to the action. As developers, we love working examples as a way to understand how things work. So, here’s how we did it.

### Prerequisites

First off, we need to have at least an Atlas account to provision Atlas and then somewhere to run our automation. You can [get an Atlas account for free](https://www.mongodb.com/cloud/atlas/register) at mongodb.com. If you want to take this demo for a spin, take the time and create your Atlas account now. Next, you’ll need to [create an organization-level API key](https://docs.atlas.mongodb.com/tutorial/manage-programmatic-access/). If you or your org already have an Atlas account you’d like to use, you’ll need the organization owner to create the organization-level API key.

Second, you’ll need an AWS account. For more information on how to create an AWS account, see [How do I create an AWS account?](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/) For this demo, we’ll be using some for-pay services like S3, but you get 12 months free. 

You will also need to have Docker installed as we are using a docker container to run all provisioning. For more information on how to install Docker, see [Get Started with Docker](https://www.docker.com/get-started). We are using Docker as it will make it easier for you to get started, as all the tooling is packaged in the container—such as AWS cli, mongosh, and Terraform.

### What You Will Build

* [MongoDB Atlas Projects](https://docs.atlas.mongodb.com/tutorial/manage-projects/) for dev, test, prod environments, to minimize blast radius in case of a human error and from a security perspective.
* [MongoDB Atlas Cluster](https://docs.atlas.mongodb.com/tutorial/create-new-cluster/) in each Atlas project (dev, test, prod). MongoDB Atlas is a fully managed data platform for modern applications. Storing data the way it is accessed as documents makes developers more productive. It provides a document-based database that is cost-efficient and resizable while automating time-consuming administration tasks such as hardware provisioning, database setup, patching, and backups. It allows you to focus on your applications by providing the foundation of high performance, high availability, security, and compatibility they need.
* [CodePipeline](https://aws.amazon.com/codepipeline/) orchestrates the CI/CD database migration stages.
* [IAM](https://aws.amazon.com/iam/) roles and policies allow cross-account access to applicable AWS resources.
* [CodeCommit](https://aws.amazon.com/codecommit/) creates a repo to store the SQL statements used when running the database migration.
* [Amazon S3](https://aws.amazon.com/s3/) creates a bucket to store pipeline artifacts.
* [CodeBuild](https://aws.amazon.com/codebuild) creates a project in target accounts using Flyway to apply database changes.
* VPC [security groups](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) ensure the secure flow of traffic between a CodeBuild project deployed within a VPC and MongoDB Atlas. AWS Private Link will also be provisioned.
* [AWS Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html) stores secrets securely and centrally, such as the Atlas API keys and database username and password.
* [Amazon SNS](https://aws.amazon.com/sns/) notifies you by email when a developer pushes changes to the CodeCommit repo.

### Step 1: Bootstrap AWS Resources

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/aws_codepipeline_161a68794c.png)

Next, we’ll fire off the script to bootstrap our AWS environment and Atlas account as shown in Diagram 1 using Terraform.

You will need to use [programmatic access keys](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html) for your AWS account and the Atlas organisation-level API key that you have created as described in the prerequisites.This is also the only time you’ll need to handle the keys manually. 

```
# Set your environment variables

# You'll find this in your Atlas console as described in prerequisites
export ATLAS_ORG_ID=60388113131271beaed5

# The public part of the Atlas Org key you created previously 
export ATLAS_ORG_PUBLIC_KEY=l3drHtms

# The private part of the Atlas Org key you created previously 
export ATLAS_ORG_PRIVATE_KEY=ab02313b-e4f1-23ad-89c9-4b6cbfa1ed4d


# Pick a username, the script will create this database user in Atlas
export DB_USER_NAME=demouser

# Pick a project base name, the script will appended -dev, -test, -prod depending on environment
export ATLAS_PROJECT_NAME=blogcicd6


# The AWS region you want to deploy into
export AWS_DEFAULT_REGION=eu-west-1


# The AWS public programmatic access key
export AWS_ACCESS_KEY_ID=AKIAZDDBLALOZWA3WWQ


# The AWS private programmatic access key
export AWS_SECRET_ACCESS_KEY=nmarrRZAIsAAsCwx5DtNrzIgThBA1t5fEfw4uJA


```

Once all the parameters are defined, you are ready to run the script that will create your CI/CD pipeline.

```
# Clone solution code repository
$ git clone https://github.com/mongodb-developer/atlas-cicd-aws
$ cd atlas-cicd

# Start docker container, which contains all the tooling e.g terraform, mongosh, and other, 
$ docker container run -it --rm -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e AWS_DEFAULT_REGION -e ATLAS_ORG_ID -e ATLAS_ORG_PUBLIC_KEY -e ATLAS_ORG_PRIVATE_KEY -e DB_USER_NAME -e ATLAS_PROJECT_NAME -v ${PWD}/terraform:/terraform piepet/cicd-mongodb:46


$ cd terraform   

# Bootstrap AWS account and Atlas Account
$ ./deploy_baseline.sh $AWS_DEFAULT_REGION $ATLAS_ORG_ID $ATLAS_ORG_PUBLIC_KEY $ATLAS_ORG_PRIVATE_KEY $DB_USER_NAME $ATLAS_PROJECT_NAME base apply




```

When deploy:baseline.sh is invoked, provisioning of AWS resources starts, using Terraform templates. The resources created are shown in Diagram 1.


From here on, you'll be able to operate your Atlas infrastructure without using your local docker instance. If you want to blaze through this guide, including cleaning it all up, you might as well keep the container running, though. The final step of tearing down the AWS infrastructure requires an external point like your local docker instance.

Until you’ve committed anything, the pipeline will have a failed Source stage. This is because it tries to check out a branch that does not exist in the code repository. After you’ve committed the Terraform code you want to execute, you’ll see that the Source stage will restart and proceed as expected. You can find the pipeline in the AWS console at this url: [https://eu-west-1.console.aws.amazon.com/codesuite/codepipeline/pipelines?region=eu-west-1](https://eu-west-1.console.aws.amazon.com/codesuite/codepipeline/pipelines?region=eu-west-1)

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/failing_pipeline_in_codepipeline_bc5629211b.png)

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/codepipeline_5ec9666ae0.png)

### Step 2: Deploy Atlas Cluster

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/deploying_an_atlas_cluster_f951a572c1.png)

Next is to deploy the Atlas cluster (projects, users, API keys, etc). This is done by pushing a configuration into the new AWS CodeCommit repo. 

If you’re like me and want to see how provisioning of the Atlas cluster works before setting up IAM properly, you can push the original github repo to AWS CodeCommit directly inside the docker container (inside the Terraform folder) using a bit of a hack. By pushing to the CodeCommit repo, AWS CodePipeline will be triggered and provisioning of the Atlas cluster will start. 

```
cd /terraform
# Push default settings to AWS Codecommit
./git_push_terraform.sh


```

To set up access to the CodeCommit repo properly, for use that survives stopping the docker container, you’ll need a proper git CodeCommit user. Follow the steps in the [AWS documentation](https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-gc.html) to create and configure your CodeCommit git user in AWS IAM. Then clone the AWS CodeCommit repository that was created in the bootstrapping, outside your docker container, perhaps in another tab in your shell, using your IAM credentials. If you did not use the “hack” to initialize it, it’ll be empty, so copy the Terraform folder that is provided in this solution, to the root of the cloned CodeCommit repository, then commit and push to kick off the pipeline. Now you can use this repo to control your setup! You should now see in the AWS CodePipeline console that the pipeline has been triggered. The pipeline will create Atlas clusters in each of the Atlas Projects and configure AWS PrivateLink. 

Let’s dive into the stages [defined in this Terraform pipeline file](https://github.com/mongodb-developer/atlas-cicd-aws/blob/main/terraform/modules/aws/code-pipeline/main.tf).

**Deploy-Base**
This is basically re-applying what we did in the bootstrapping. This stage ensures we can improve on the AWS pipeline infrastructure itself over time.

This stage creates the projects in Atlas, including Atlas project API keys, Atlas project users, and database users. 

**Deploy-Dev**

This stage creates the corresponding Private Link and MongoDB cluster.

**Deploy-Test**

This stage creates the corresponding Private Link and MongoDB cluster.

**Deploy-Prod**

This stage creates the corresponding Private Link and MongoDB cluster.

**Gate**

Approving means we think it all looks good. Perhaps counter intuitively but great for demos, it proceeds to teardown. This might be one of the first behaviours you’ll change. :)

**Teardown**

This decommissions the dev, test, and prod resources we created above. To decommission the base resources, including the pipeline itself, we recommend you run that externally—for example, from the Docker container on your laptop. We’ll cover that later.

As you advance towards the Gate stage, you’ll see the Atlas clusters build out. Below is an example where the Test stage is creating a cluster. Approving the Gate will undeploy the resources created in the dev, test, and prod stages, but keep projects and users.

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/project_names_and_clusters_041271acd0.png)

### Step 3: Make a Change!

Assuming you took the time to set up IAM properly, you can now work with the infrastructure as code directly from your laptop outside the container. If you just deployed using the hack inside the container, you can continue interacting using the repo created inside the Docker container, but at some point, the container will stop and that repo will be gone. So, beware.

Navigate to the root of the clone of the CodeCommit repo. For example, if you used the script in the container, you’d run, also in the container:

```
cd /${ATLAS_PROJECT_NAME}-base-repo/

```

Then you can edit, for example, the MongoDB version by changing 4.4 to 5.0 in `terraform/environment/dev/variables.tf`.

```
variable "cluster_mongodbversion" {
 description = "The Major MongoDB Version"
 default = "5.0"
}
```

Then push (git add, commit, push) and you’ll see a new run initiated in CodePipeline.

### Step 4: Clean Up Base Infrastructure

Now, that was interesting. Time for cleaning up! To decommission the full environment, you should first approve the Gate stage to execute the teardown job. When that’s been done, only the base infrastructure remains. Start the container again as in Step 1 if it’s not running, and then execute deploy_baseline.sh, replacing the word ***apply*** with ***destroy***: 

```
# inside the /terraform folder of the container

# Clean up AWS and Atlas Account
./deploy_baseline.sh $AWS_DEFAULT_REGION $ATLAS_ORG_ID $ATLAS_ORG_PUBLIC_KEY $ATLAS_ORG_PRIVATE_KEY $DB_USER_NAME $ATLAS_PROJECT_NAME base destroy

```

## Lessons Learned

In this solution, we have separated the creation of AWS resources and the Atlas cluster, as the changes to the Atlas cluster will be more frequent than the changes to the AWS resources. 

When implementing infrastructure as code for a MongoDB Atlas Cluster, you have to consider not just the cluster creation but also a strategy for how to separate dev, qa, and prod environments and how to store secrets. This to minimize blast radius. 

We also noticed how useful resource tagging is to make Terraform scripts portable. By setting tags on AWS resources, the script does not need to know the names of the resources but can look them up by tag instead.

## Conclusion

By using CI/CD automation for Atlas clusters, you can speed up deployments and increase the agility of your software teams. 

MongoDB Atlas offers a powerful API that, in combination with AWS CI/CD services and Terraform, can support continuous delivery of MongoDB Atlas clusters, and version-control the database lifecycle. You can apply the same pattern with other CI/CD tools that aren’t specific to AWS. 

In this blog, we’ve offered an exhaustive, reproducible, and reusable deployment process for MongoDB Atlas, including traceability. A devops team can use our demonstration as inspiration for how to quickly deploy MongoDB Atlas, automatically embedding organisation best practices. 

In this blog, we’ll demonstrate how to set up CI/CD for MongoDB Atlas, in a typical production setting.

Building a Multi-Environment Continuous Delivery Pipeline for MongoDB Atlas

Rachelle Palmer

Looking for a code example that has microservices in Go with Docker, Kubernetes and MongoDB? Look no further!

Cinema is an example project which demonstrates the use of microservices for a fictional movie theater. The Cinema backend is powered by 4 microservices, all of which happen to be written in Go, using MongoDB for manage the database and Docker to isolate and deploy the ecosystem.

Movie Service: Provides information like movie ratings, title, etc.
Show Times Service: Provides show times information.
Booking Service: Provides booking information.
Users Service: Provides movie suggestions for users by communicating with other services.

This project is available to clone or fork on [github](https://github.com/mmorejon/microservices-docker-go-mongodb) from [Manuel Morejón](https://twitter.com/morejon85). 


 An easy project using Go, Docker, Kubernetes and MongoDB

Cinema: Example Go Microservices Application

Stanimira Vlaeva

Abirami Sukumaran

## Plea and the Pledge: Truly Serverless

As modern application developers, we’re juggling many priorities: performance, flexibility, usability, security, reliability, and maintainability. On top of that, we’re handling dependencies, configuration, and deployment of multiple components in multiple environments and sometimes multiple repositories as well. And then we have to keep things secure and simple. Ah, the nightmare!

This is the reason we love serverless computing. Serverless allows developers to focus on the thing they like to do the most—development—and leave the rest of the attributes, including infrastructure and maintenance, to the platform offerings. 

In this read, we’re going to see how Cloud Run and MongoDB come together to enable a completely serverless MEAN stack application development experience. We'll learn how to build a serverless MEAN application with [Cloud Run](https://cloud.google.com/run/) and [MongoDB Atlas](https://www.mongodb.com/atlas), the multi-cloud application data platform by MongoDB.

### Containerized deployments with Cloud Run

All serverless platform offer exciting capabilities: 

* Event-driven function (not a hard requirement though)
* No-infrastructure maintenance
* Usage-based pricing
* Auto-scaling capabilities

Cloud Run stands out of the league by enabling us to: 

* Package code in multiple stateless containers that are request-aware and invoke it via HTTP requests
* Only be charged for the exact resources you use
* Support any programming language or any operating system library of your choice, or any binary

Check this link for more features in full [context](https://cloud.google.com/run/?utm_source=google&utm_medium=cpc&utm_campaign=japac-IN-all-en-dr-skws-all-pkws-trial-b-dr-1009882&utm_content=text-ad-none-none-DEV_c-CRE_529584319423-ADGP_Hybrid%20%7C%20SKWS%20-%20PHR%20%7C%20Txt%20~%20Compute%20~%20Cloud%20Run_Global%20gap%20analysis-KWID_43700065767178853-aud-970366092687%3Akwd-677335471899&userloc_1007768-network_g&utm_term=KW_cloud%20run&gclid=Cj0KCQjw1tGUBhDXARIsAIJx01lcrrpCE3MLqe4k3MiQdd8ibvGmXduL9kLXELPjodjSu7Gipc5ODYwaAlBkEALw_wcB&gclsrc=aw.ds).

However, many serverless models overlook the fact that traditional databases are not managed. You need to manually provision infrastructure (vertical scaling) or add more servers (horizontal scaling) to scale the database. This introduces a bottleneck in your serverless architecture and can lead to performance issues.

### Deploy a serverless database with MongoDB Atlas

MongoDB launched serverless instances, a new fully managed, serverless database deployment in Atlas to solve this problem. With serverless instances you never have to think about infrastructure — simply deploy your database and it will scale up and down seamlessly based on demand — requiring no hands-on management. And the best part, you will only be charged for the operations you run. To make our architecture truly serverless, we'll combine Cloud Run and MongoDB Atlas capabilities.

## What's the MEAN stack?

The [MEAN stack](https://www.mongodb.com/mean-stack) is a technology stack for building full-stack web applications entirely with JavaScript and JSON. The MEAN stack is composed of four main components—MongoDB, Express, Angular, and Node.js.

* **MongoDB** is responsible for data storage. 
* **Express.js** is a Node.js web application framework for building APIs.
* **Angular** is a client-side JavaScript platform.
* **Node.js** is a server-side JavaScript runtime environment. The server uses the MongoDB Node.js driver to connect to the database and retrieve and store data. 

![MEAN stack](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/mean_stack_e791a7354e.PNG)

## Steps for deploying truly serverless MEAN stack apps with Cloud Run and MongoDB

In the following sections, we’ll provision a new MongoDB serverless instance, connect a MEAN stack web application to it, and finally, deploy the application to Cloud Run.

### 1. Create the database

Before you begin, get started with [MongoDB Atlas on Google Cloud](https://console.cloud.google.com/marketplace/product/mongodb/mdb-atlas-self-service).

![MongoDB Atlas on Google Cloud subscription page](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/mongodb_atlas_on_google_cloud_e3bcc0b537.JPG)

Once you sign up, click the “Build a Database” button to create a new serverless instance. Select the following configuration:

![Create new serverless instance in MongoDB Atlas configuration page
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/create_new_serverless_instance_9134772726.JPG)

Once your serverless instance is provisioned, you should see it up and running.

![Database deployments page in MongoDB Atlas
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/database_deployments_page_5309d13fb4.JPG)

Click on the “Connect” button to add a connection IP address and a database user.

For this blog post, we’ll use the “Allow Access from Anywhere” setting. MongoDB Atlas comes with a set of security and access features. You can learn more about them in the [security features documentation article](https://www.mongodb.com/docs/atlas/setup-cluster-security/).

Use credentials of your choice for the database username and password. Once these steps are complete, you should see the following:

![Screen showing 'You are ready to connect' in MongoDB Atlas
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/ready_to_connect_e06352f53b.JPG)

Proceed by clicking on the “Choose a connection method” button and then selecting “Connect your application”.

![Screen showing the connection string for MongoDB Atlas
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/connection_string_mongodb_atlas_c28ea136ce.JPG)

Copy the connection string you see and replace the password with your own. We’ll use that string to connect to our database in the following sections.

### 2. Set up a Cloud Run project

First, sign in to [Cloud Console](http://console.cloud.google.com/), create a new project, or reuse an existing one.

Remember the Project Id for the project you created. Below is an image from [https://codelabs.developers.google.com/codelabs/cloud-run-hello#1](https://codelabs.developers.google.com/codelabs/cloud-run-hello#1) that shows how to create a new project in Google Cloud.

!['Create a new project' configuration page in Google Cloud. The project id is highlighted.
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/create_new_project_605bbeec53.JPG)

Then, enable Cloud Run API from [Cloud Shell](https://cloud.google.com/shell):

* Activate Cloud Shell from the Cloud Console. Simply click Activate Cloud Shell.

![Google Cloud Shell Editor loading page
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/google_cloud_shell_editor_18236232e8.JPG)

* Use the below command:

*gcloud services enable run.googleapis.com*

We will be using Cloud Shell and [Cloud Shell Editor](https://cloud.google.com/shell/docs/editor-overview#:~:text=Cloud%20Shell%20comes%20with%20a,and%20is%20based%20on%20Theia.) for code references. To access Cloud Shell Editor, click Open Editor from the Cloud Shell Terminal:

![Google Cloud Shell Code Editor
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/google_cloud_shell_code_editor_51ed2f16ca.JPG)

Finally, we need to clone the MEAN stack project we’ll be deploying. 

We’ll deploy an employee management web application. The REST API is built with Express and Node.js; the web interface, with Angular; and the data will be stored in the MongoDB Atlas instance we created earlier.

Clone the project repository by executing the following command in the Cloud Shell Terminal:

`git clone` [https://github.com/mongodb-developer/mean-stack-example.git](https://github.com/mongodb-developer/mean-stack-example.git)

In the following sections, we will deploy a couple of services—one for the Express REST API and one for the Angular web application. 

### 3. Deploy the Express and Node.js REST API

First, we’ll deploy a Cloud Run service for the Express REST API. 

The most important file for our deployment is the Docker configuration file. Let’s take a look at it:

**mean-stack-example/server/Dockerfile**

```
FROM node:17-slim
 
WORKDIR /usr/app
COPY ./ /usr/app
 
# Install dependencies and build the project.
RUN npm install
RUN npm run build
 
# Run the web service on container startup.
CMD ["node", "dist/server.js"]
```

The configuration sets up Node.js, and copies and builds the project. When the container starts, the command “node dist/server.js” starts the service.

To start a new Cloud Run deployment, click on the Cloud Run icon on the left sidebar:

![Select the 'Cloud Run' icon from the left sidebar
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/select_new_cloud_run_e2c48b457a.JPG)

Then, click on the Deploy to Cloud Run icon:

![Select the 'Deploy to Cloud Run' icon from the opened pane
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/select_deploy_to_cloud_run_c90b1dc8f6.JPG)

Fill in the service configuration as follows:

* Service name: node-express-api
* Deployment platform: Cloud Run (fully managed)
* Region: Select a region close to your database region to reduce latency
* Authentication: Allow unauthenticated invocations

Under Revision Settings, click on Show Advanced Settings to expand them:

* Container port: 5200
* Environment variables. Add the following key-value pair and make sure you add the connection string for your own MongoDB Atlas deployment:

`ATLAS_URI:mongodb+srv:/<username>:<password>@sandbox.pv0l7.mongodb.net/meanStackExample?retryWrites=true&w=majority`

For the Build environment, select Cloud Build.

Finally, in the Build Settings section, select:

* Builder: Docker
* Docker: mean-stack-example/server/Dockerfile

Click the Deploy button and then Show Detailed Logs to follow the deployment of your first Cloud Run service!

After the build has completed, you should see the URL of the deployed service:

![Screenshot displaying the message 'Deployment completed successfully!' and the deployment URL.
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/deployment_completed_successfully_31bcb13b6f.JPG)

Open the URL and append ‘/employees’ to the end. You should see an empty array because currently, there are no documents in the database. Let’s deploy the user interface so we can add some!

### 4. Deploy the Angular web application

Our Angular application is in the client directory. To deploy it, we’ll use the Nginx server and Docker.

> Just a thought, there is also an option to use Firebase Hosting for your [Angular application deployment](https://www.youtube.com/watch?v=jVPHsolXgLk) as you can serve your content to a CDN (content delivery network) directly.

Let’s take a look at the configuration files:

**mean-stack-example/client/nginx.conf**

```
events{}
 
http {
 
   include /etc/nginx/mime.types;
 
   server {
       listen 8080;
       server_name 0.0.0.0;
       root /usr/share/nginx/html;
       index index.html;
 
       location / {
           try_files $uri $uri/ /index.html;
       }
   }
}
```

In the Nginx configuration, we specify the default port—8080, and the starting file—`index.html`.

**mean-stack-example/client/Dockerfile**

```
FROM node:17-slim AS build
 
WORKDIR /usr/src/app
COPY package.json package-lock.json ./
 
# Install dependencies and copy them to the container
RUN npm install
COPY . .
 
# Build the Angular application for production
RUN npm run build --prod
 
# Configure the nginx web server
FROM nginx:1.17.1-alpine
COPY nginx.conf /etc/nginx/nginx.conf
COPY --from=build /usr/src/app/dist/client /usr/share/nginx/html
 
# Run the web service on container startup.
CMD ["nginx", "-g", "daemon off;"]
```

In the Docker configuration, we install Node.js dependencies and build the project. Then, we copy the built files to the container, configure, and start the Nginx service.

Finally, we need to configure the URL to the REST API so that our client application can send requests to it. Since we’re only using the URL in a single file in the project, we’ll hardcode the URL. Alternatively, you can attach the environment variable to the window object and access it from there.

**mean-stack-example/client/src/app/employee.service.ts**

```
@Injectable({
 providedIn: 'root'
})
export class EmployeeService {
 // Replace with the URL of your REST API
 private url = 'https://node-express-api-vsktparjta-uc.a.run.app'; 
…
```

We’re ready to deploy to Cloud Run! Start a new deployment with the following configuration settings:

* Service Settings: Create a service
* Service name: angular-web-app
* Deployment platform: Cloud Run (fully managed)
* Authentication: Allow unauthenticated invocations

For the Build environment, select Cloud Build.

Finally, in the Build Settings section, select:

* Builder: Docker
* Docker: mean-stack-example/client/Dockerfile

Click that Deploy button again and watch the logs as your app is shipped to the cloud! When the deployment is complete, you should see the URL for the client app:

![Screenshot displaying the message 'Deployment completed successfully!' and the deployment URL for the Angular service.
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/deployment_completed_successfully_notice_6602368f53.JPG)

Open the URL, and play with your application!

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/employees_demo_1_da0a6342ba.gif)

### Command shell alternative for build and deploy

The steps covered above can alternatively be implemented from Command Shell as below:

Step 1: Create the new project directory named “mean-stack-example” either from the Code Editor or Cloud Shell Command (Terminal):

*mkdir mean-stack-demo
cd mean-stack-demo*

Step 2: Clone project repo and make necessary changes in the configuration and variables, same as mentioned in the previous section.

Step 3: Build your container image using Cloud build by running the command in Cloud Shell:

*gcloud builds submit --tag gcr.io/$GOOGLECLOUDPROJECT/mean-stack-demo*

$GOOGLE_CLOUD_PROJECT is an environment variable containing your Google Cloud project ID when running in Cloud Shell.

Step 4: Test it locally by running: 
docker run -d -p 8080:8080 gcr.io/$GOOGLE_CLOUD_PROJECT/mean-stack-demo 
and by clicking Web Preview, Preview on port 8080.

Step 5: Run the following command to deploy your containerized app to Cloud Run:

*gcloud run deploy mean-stack-demo --image 
gcr.io/$GOOGLECLOUDPROJECT/mean-stack-demo --platform managed --region us-central1 --allow-unauthenticated --update-env-vars DBHOST=$DB_HOST*

a. –allow-unauthenticated will let the service be reached without authentication.

b. –platform-managed means you are requesting the fully managed environment and not the Kubernetes one via Anthos.
 
c. –update-env-vars expects the MongoDB Connection String to be passed on to the environment variable DBHOST.
Hang on until the section on Env variable and Docker for Continuous Deployment for Secrets and Connection URI management.

d. When the deployment is done, you should see the deployed service URL in the command line.

e. When you hit the service URL, you should see your web page on the browser and the logs in the Cloud Logging Logs Explorer page. 

![Service logs displayed in the Cloud Logging Logs Explorer page
](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/services_logs_2085273e8a.JPG)

### 5. Environment variables and Docker for continuous deployment

If you’re looking to automate the process of building and deploying across multiple containers, services, or components, storing these configurations in the repo is not only cumbersome but also a security threat. 

1. For ease of cross-environment continuous deployment and to avoid security vulnerabilities caused by leaking credential information, we can choose to pass variables at build/deploy/up time.
    
    *--update-env-vars* allows you to set the environment variable to a value that is passed only at run time. In our example, the variable DBHOST is assigned the value of $DB_HOST. which is set as *DB_HOST = ‘<<ENCODED CONNECTION URI>>’*.

    Please note that unencoded symbols in Connection URI (username, password) will result in connection issues with MongoDB. For example, if you have a $ in the password or username, replace it with %24 in the encoded Connection URI.

2. Alternatively, you can also pass configuration variables as env variables at build time into docker-compose (*docker-compose.yml*). By passing configuration variables and credentials, we avoid credential leakage and automate deployment securely and continuously across multiple environments, users, and applications.

## Conclusion

MongoDB Atlas with Cloud Run makes for a truly serverless MEAN stack solution, and for those looking to build an application with a serverless option to run in a stateless container, Cloud Run is your best bet. 

## Before you go…

Now that you have learnt how to deploy a simple MEAN stack application on Cloud Run and MongoDB Atlas, why don’t you take it one step further with your favorite client-server use case? Reference the below resources for more inspiration:

* Cloud Run HelloWorld: [https://codelabs.developers.google.com/codelabs/cloud-run-hello#4](https://codelabs.developers.google.com/codelabs/cloud-run-hello#4)
* MongoDB - MEAN Stack: [https://www.mongodb.com/languages/mean-stack-tutorial](https://www.mongodb.com/languages/mean-stack-tutorial)

If you have any comments or questions, feel free to reach out to us online: [Abirami Sukumaran](http://www.linkedin.com/in/abiramisukumaran) and [Stanimira Vlaeva](https://twitter.com/StanimiraVlaeva).

In this blog, we'll see how Cloud Run and MongoDB come together to enable a completely serverless MEAN stack application development experience.

Docker

All Docker Content

Related Topics