Johannes Brännström

Senior Solutions Architect

Pierre Petersson

Advisory Solutions Architect

## Why CI/CD?

To increase the speed and quality of development, you may use continuous delivery strategies to manage and deploy your application code changes. However, continuous delivery for databases is often a manual process.

Adopting continuous integration and continuous delivery (CI/CD) for managing the lifecycle of a database has the following benefits:

* An automated multi-environment setup enables you to move faster and focus on what really matters.
* The confidence level of the changes applied increases.
* The process is easier to reproduce.
* All changes to database configuration will be traceable.

### Why CI/CD for MongoDB Atlas?

[MongoDB Atlas](https://www.mongodb.com/cloud/atlas) is a multi-cloud developer data platform, providing an integrated suite of cloud database and data services to accelerate and simplify how you build with data. MongoDB Atlas also provides a comprehensive [API](https://docs.atlas.mongodb.com/api/atlas-admin-api/), making CI/CD for the actual data platform itself possible.

In this blog, we’ll demonstrate how to set up CI/CD for MongoDB Atlas, in a typical production setting. The intended audience is developers, solutions architects, and database administrators with knowledge of MongoDB Atlas, AWS, and Terraform.

## Our CI/CD Solution Requirements

* Ensure that each environment (dev, test, prod) is isolated to minimize blast radius in case of a human error or from a security perspective. MongoDB Atlas Projects and API Keys will be utilized to enable environment isolation.
* All services used in this solution will use managed services. This to minimize the time needed to spend on managing infrastructure.
* Minimize commercial agreements required. Use as much as possible from AWS and the Atlas ecosystem so that there is no need to purchase external tooling, such as HashiCorp Vault. 
* Minimize time spent on installing local dev tooling, such as git and Terraform. The solution will provide a docker image, with all tooling required to run provisioning of Terraform templates. The same image will be used to also run the pipeline in AWS CodeBuild. 

## Implementation

Enough talk—let’s get to the action. As developers, we love working examples as a way to understand how things work. So, here’s how we did it.

### Prerequisites

First off, we need to have at least an Atlas account to provision Atlas and then somewhere to run our automation. You can [get an Atlas account for free](https://www.mongodb.com/cloud/atlas/register) at mongodb.com. If you want to take this demo for a spin, take the time and create your Atlas account now. Next, you’ll need to [create an organization-level API key](https://docs.atlas.mongodb.com/tutorial/manage-programmatic-access/). If you or your org already have an Atlas account you’d like to use, you’ll need the organization owner to create the organization-level API key.

Second, you’ll need an AWS account. For more information on how to create an AWS account, see [How do I create an AWS account?](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/) For this demo, we’ll be using some for-pay services like S3, but you get 12 months free. 

You will also need to have Docker installed as we are using a docker container to run all provisioning. For more information on how to install Docker, see [Get Started with Docker](https://www.docker.com/get-started). We are using Docker as it will make it easier for you to get started, as all the tooling is packaged in the container—such as AWS cli, mongosh, and Terraform.

### What You Will Build

* [MongoDB Atlas Projects](https://docs.atlas.mongodb.com/tutorial/manage-projects/) for dev, test, prod environments, to minimize blast radius in case of a human error and from a security perspective.
* [MongoDB Atlas Cluster](https://docs.atlas.mongodb.com/tutorial/create-new-cluster/) in each Atlas project (dev, test, prod). MongoDB Atlas is a fully managed data platform for modern applications. Storing data the way it is accessed as documents makes developers more productive. It provides a document-based database that is cost-efficient and resizable while automating time-consuming administration tasks such as hardware provisioning, database setup, patching, and backups. It allows you to focus on your applications by providing the foundation of high performance, high availability, security, and compatibility they need.
✅ Already have an AWS account? Atlas supports paying for usage via the AWS Marketplace (AWS MP) without any upfront commitment — simply <a href="https://aws.amazon.com/marketplace/pp/prodview-pp445qepfdy34?trk=38088f03-4928-4b00-9a39-86781df6ab9b&sc_channel=el" target="_blank" rel="noreferrer">sign up for MongoDB Atlas via AWS Marketplace.<a>
* [CodePipeline](https://aws.amazon.com/codepipeline/) orchestrates the CI/CD database migration stages.
* [IAM](https://aws.amazon.com/iam/) roles and policies allow cross-account access to applicable AWS resources.
* [CodeCommit](https://aws.amazon.com/codecommit/) creates a repo to store the SQL statements used when running the database migration.
* [Amazon S3](https://aws.amazon.com/s3/) creates a bucket to store pipeline artifacts.
* [CodeBuild](https://aws.amazon.com/codebuild) creates a project in target accounts using Flyway to apply database changes.
* VPC [security groups](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) ensure the secure flow of traffic between a CodeBuild project deployed within a VPC and MongoDB Atlas. AWS Private Link will also be provisioned.
* [AWS Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html) stores secrets securely and centrally, such as the Atlas API keys and database username and password.
* [Amazon SNS](https://aws.amazon.com/sns/) notifies you by email when a developer pushes changes to the CodeCommit repo.

### Step 1: Bootstrap AWS Resources

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/aws_codepipeline_161a68794c.png)

Next, we’ll fire off the script to bootstrap our AWS environment and Atlas account as shown in Diagram 1 using Terraform.

You will need to use [programmatic access keys](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html) for your AWS account and the Atlas organisation-level API key that you have created as described in the prerequisites.This is also the only time you’ll need to handle the keys manually. 

```
# Set your environment variables

# You'll find this in your Atlas console as described in prerequisites
export ATLAS_ORG_ID=60388113131271beaed5

# The public part of the Atlas Org key you created previously 
export ATLAS_ORG_PUBLIC_KEY=l3drHtms

# The private part of the Atlas Org key you created previously 
export ATLAS_ORG_PRIVATE_KEY=ab02313b-e4f1-23ad-89c9-4b6cbfa1ed4d


# Pick a username, the script will create this database user in Atlas
export DB_USER_NAME=demouser

# Pick a project base name, the script will appended -dev, -test, -prod depending on environment
export ATLAS_PROJECT_NAME=blogcicd6


# The AWS region you want to deploy into
export AWS_DEFAULT_REGION=eu-west-1


# The AWS public programmatic access key
export AWS_ACCESS_KEY_ID=AKIAZDDBLALOZWA3WWQ


# The AWS private programmatic access key
export AWS_SECRET_ACCESS_KEY=nmarrRZAIsAAsCwx5DtNrzIgThBA1t5fEfw4uJA


```

Once all the parameters are defined, you are ready to run the script that will create your CI/CD pipeline.

```
# Clone solution code repository
$ git clone https://github.com/mongodb-developer/atlas-cicd-aws
$ cd atlas-cicd

# Start docker container, which contains all the tooling e.g terraform, mongosh, and other, 
$ docker container run -it --rm -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e AWS_DEFAULT_REGION -e ATLAS_ORG_ID -e ATLAS_ORG_PUBLIC_KEY -e ATLAS_ORG_PRIVATE_KEY -e DB_USER_NAME -e ATLAS_PROJECT_NAME -v ${PWD}/terraform:/terraform piepet/cicd-mongodb:46


$ cd terraform   

# Bootstrap AWS account and Atlas Account
$ ./deploy_baseline.sh $AWS_DEFAULT_REGION $ATLAS_ORG_ID $ATLAS_ORG_PUBLIC_KEY $ATLAS_ORG_PRIVATE_KEY $DB_USER_NAME $ATLAS_PROJECT_NAME base apply




```

When deploy:baseline.sh is invoked, provisioning of AWS resources starts, using Terraform templates. The resources created are shown in Diagram 1.


From here on, you'll be able to operate your Atlas infrastructure without using your local docker instance. If you want to blaze through this guide, including cleaning it all up, you might as well keep the container running, though. The final step of tearing down the AWS infrastructure requires an external point like your local docker instance.

Until you’ve committed anything, the pipeline will have a failed Source stage. This is because it tries to check out a branch that does not exist in the code repository. After you’ve committed the Terraform code you want to execute, you’ll see that the Source stage will restart and proceed as expected. You can find the pipeline in the AWS console at this url: [https://eu-west-1.console.aws.amazon.com/codesuite/codepipeline/pipelines?region=eu-west-1](https://eu-west-1.console.aws.amazon.com/codesuite/codepipeline/pipelines?region=eu-west-1)

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/failing_pipeline_in_codepipeline_bc5629211b.png)

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/codepipeline_5ec9666ae0.png)

### Step 2: Deploy Atlas Cluster

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/deploying_an_atlas_cluster_f951a572c1.png)

Next is to deploy the Atlas cluster (projects, users, API keys, etc). This is done by pushing a configuration into the new AWS CodeCommit repo. 

If you’re like me and want to see how provisioning of the Atlas cluster works before setting up IAM properly, you can push the original github repo to AWS CodeCommit directly inside the docker container (inside the Terraform folder) using a bit of a hack. By pushing to the CodeCommit repo, AWS CodePipeline will be triggered and provisioning of the Atlas cluster will start. 

```
cd /terraform
# Push default settings to AWS Codecommit
./git_push_terraform.sh


```

To set up access to the CodeCommit repo properly, for use that survives stopping the docker container, you’ll need a proper git CodeCommit user. Follow the steps in the [AWS documentation](https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-gc.html) to create and configure your CodeCommit git user in AWS IAM. Then clone the AWS CodeCommit repository that was created in the bootstrapping, outside your docker container, perhaps in another tab in your shell, using your IAM credentials. If you did not use the “hack” to initialize it, it’ll be empty, so copy the Terraform folder that is provided in this solution, to the root of the cloned CodeCommit repository, then commit and push to kick off the pipeline. Now you can use this repo to control your setup! You should now see in the AWS CodePipeline console that the pipeline has been triggered. The pipeline will create Atlas clusters in each of the Atlas Projects and configure AWS PrivateLink. 

Let’s dive into the stages [defined in this Terraform pipeline file](https://github.com/mongodb-developer/atlas-cicd-aws/blob/main/terraform/modules/aws/code-pipeline/main.tf).

**Deploy-Base**
This is basically re-applying what we did in the bootstrapping. This stage ensures we can improve on the AWS pipeline infrastructure itself over time.

This stage creates the projects in Atlas, including Atlas project API keys, Atlas project users, and database users. 

**Deploy-Dev**

This stage creates the corresponding Private Link and MongoDB cluster.

**Deploy-Test**

This stage creates the corresponding Private Link and MongoDB cluster.

**Deploy-Prod**

This stage creates the corresponding Private Link and MongoDB cluster.

**Gate**

Approving means we think it all looks good. Perhaps counter intuitively but great for demos, it proceeds to teardown. This might be one of the first behaviours you’ll change. :)

**Teardown**

This decommissions the dev, test, and prod resources we created above. To decommission the base resources, including the pipeline itself, we recommend you run that externally—for example, from the Docker container on your laptop. We’ll cover that later.

As you advance towards the Gate stage, you’ll see the Atlas clusters build out. Below is an example where the Test stage is creating a cluster. Approving the Gate will undeploy the resources created in the dev, test, and prod stages, but keep projects and users.

![](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/project_names_and_clusters_041271acd0.png)

### Step 3: Make a Change!

Assuming you took the time to set up IAM properly, you can now work with the infrastructure as code directly from your laptop outside the container. If you just deployed using the hack inside the container, you can continue interacting using the repo created inside the Docker container, but at some point, the container will stop and that repo will be gone. So, beware.

Navigate to the root of the clone of the CodeCommit repo. For example, if you used the script in the container, you’d run, also in the container:

```
cd /${ATLAS_PROJECT_NAME}-base-repo/

```

Then you can edit, for example, the MongoDB version by changing 4.4 to 5.0 in `terraform/environment/dev/variables.tf`.

```
variable "cluster_mongodbversion" {
 description = "The Major MongoDB Version"
 default = "5.0"
}
```

Then push (git add, commit, push) and you’ll see a new run initiated in CodePipeline.

### Step 4: Clean Up Base Infrastructure

Now, that was interesting. Time for cleaning up! To decommission the full environment, you should first approve the Gate stage to execute the teardown job. When that’s been done, only the base infrastructure remains. Start the container again as in Step 1 if it’s not running, and then execute deploy_baseline.sh, replacing the word ***apply*** with ***destroy***: 

```
# inside the /terraform folder of the container

# Clean up AWS and Atlas Account
./deploy_baseline.sh $AWS_DEFAULT_REGION $ATLAS_ORG_ID $ATLAS_ORG_PUBLIC_KEY $ATLAS_ORG_PRIVATE_KEY $DB_USER_NAME $ATLAS_PROJECT_NAME base destroy

```

## Lessons Learned

In this solution, we have separated the creation of AWS resources and the Atlas cluster, as the changes to the Atlas cluster will be more frequent than the changes to the AWS resources. 

When implementing infrastructure as code for a MongoDB Atlas Cluster, you have to consider not just the cluster creation but also a strategy for how to separate dev, qa, and prod environments and how to store secrets. This to minimize blast radius. 

We also noticed how useful resource tagging is to make Terraform scripts portable. By setting tags on AWS resources, the script does not need to know the names of the resources but can look them up by tag instead.

## Conclusion

By using CI/CD automation for Atlas clusters, you can speed up deployments and increase the agility of your software teams. 

MongoDB Atlas offers a powerful API that, in combination with AWS CI/CD services and Terraform, can support continuous delivery of MongoDB Atlas clusters, and version-control the database lifecycle. You can apply the same pattern with other CI/CD tools that aren’t specific to AWS. 

In this blog, we’ve offered an exhaustive, reproducible, and reusable deployment process for MongoDB Atlas, including traceability. A devops team can use our demonstration as inspiration for how to quickly deploy MongoDB Atlas, automatically embedding organisation best practices. 

In this blog, we’ll demonstrate how to set up CI/CD for MongoDB Atlas, in a typical production setting.

Building a Multi-Environment Continuous Delivery Pipeline for MongoDB Atlas

In this blog, learn how to use Hashicorp Vault KMIP Secrets Engine with CSFLE and Queryable Encryption to have a standardized way of managing encryption keys.

How to Set Up HashiCorp Vault KMIP Secrets Engine with MongoDB CSFLE or Queryable Encryption

Tom McCarthy

The right to erasure, also known as the right to be forgotten, is a right granted to individuals under laws and regulations such as GDPR. This means that companies storing an individual's personal data must be able to delete it on request. Because this data can be spread across several systems, it can be technically challenging for these companies to identify and remove it from all places. Even if this is properly executed, there is also a risk that deleted data can be restored from backups in the future, potentially contributing to legal and financial risks.

This blog post addresses those challenges, demonstrating how you can make use of MongoDB's Client-Side Field Level Encryption to strengthen procedures for removing sensitive data.

>***Disclaimer**: We provide no guarantees that the solution and techniques described in this article will fulfill regulatory requirements around the right to erasure. Each organization needs to make their own determination on appropriate or sufficient measures to comply with various regulatory requirements such as GDPR.*

## What is crypto shredding?
Crypto shredding is a data destruction technique that consists of destroying the encryption keys that allow the data to be decrypted, thus making the data undecipherable. The example below gives a more in-depth explanation.

Imagine you are storing data for multiple users. You start by giving each user their own unique data encryption key (DEK), and mapping it to that customer. This is represented in the below diagram, where "User A" and "User B" each have their own key in the key store. This DEK can then be used to encrypt and decrypt any data related to the user in question.

![Diagram showing different data encryption keys for User A and User B](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/data_encryption_keys_602f207e54.png)

Let's assume that we want to remove all data for User B. If we remove User B's DEK, we can no longer decrypt any of the data that was encrypted with it; all we have left in our data store is "junk" cipher text. As the diagram below illustrates, User A's data is unaffected, but we can no longer read User B's data.

![Diagram showing what happens when user B's data encryption key is deleted from the key store](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/encryption_key_deleted_8c803d3e52.png)

## What is CSFLE?
With MongoDB’s [Client-Side Field Level Encryption](https://www.mongodb.com/docs/manual/core/csfle/) (CSFLE), applications can encrypt sensitive fields in documents prior to transmitting data to the server. This means that even when data is being used by the database in memory, it is never in plain text. The database only sees the encrypted data but still enables you to query it.

MongoDB CSFLE utilizes envelope encryption, which is the practice of encrypting plaintext data with a data key, which itself is in turn encrypted by a top level envelope key (also known as a "master key"). 

![Diagram showing an envelope key being used to encrypt a data key, which in turn encrypts data](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/envelop_key_6a599cfd2e.png)

Envelope keys are usually managed by a Key Management Service (KMS). MongoDB CSFLE supports multiple KMSs, such as AWS KMS, GCP KMS, Azure KeyVault, and Keystores supporting the KMIP standard (e.g., Hashicorp Keyvault).

CSFLE can be used in either [automatic](https://www.mongodb.com/docs/manual/core/csfle/fundamentals/automatic-encryption/#std-label-csfle-fundamentals-automatic-encryption) mode or [explicit mode](https://www.mongodb.com/docs/manual/core/csfle/fundamentals/manual-encryption/#std-label-csfle-fundamentals-manual-encryption) — or a combination of both. Automatic mode enables you to perform encrypted read and write operations based on a defined encryption schema, avoiding the need for application code to specify how to encrypt or decrypt fields. This [encryption schema](https://www.mongodb.com/docs/manual/core/csfle/fundamentals/create-schema/#std-label-csfle-fundamentals-create-schema) is a JSON document that defines what fields need to be encrypted. [Explicit](https://www.mongodb.com/docs/manual/core/csfle/fundamentals/manual-encryption/#std-label-csfle-fundamentals-manual-encryption) mode refers to using the MongoDB driver's encryption library to manually encrypt or decrypt fields in your application.

In this article, we are going to use the [explicit encryption](https://www.mongodb.com/docs/manual/core/csfle/fundamentals/manual-encryption/#std-label-csfle-fundamentals-manual-encryption) technique to showcase how we can use crypto shredding techniques with CSFLE to implement (or augment) procedures to "forget" sensitive data. We'll be using AWS KMS to demonstrate this.

## Bringing it all together
With MongoDB as our database, we can use CSFLE to implement crypto shredding, so we can provide stronger guarantees around data privacy.

To demonstrate how you could implement this, we'll walk you through a demo application. The demo application is a python (Flask) web application with a front end, which exposes functionality for signup, login, and a data entry form. We have also added an "admin" page to showcase the crypto shredding related functionality. If you want to follow along, you can run the application yourself — you'll find the necessary code and instructions in [GitHub](https://github.com/mongodb-developer/mongodb-flask-cryptoshredding-example).

![demo application home page](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/demo_application_homepage_ad48ff5c3d.png)

When a user signs up, our application will generate a DEK for the user, then store the ID for the DEK along with other user details. Key generation is done via the  `create_data_key` method on the `ClientEncryption` class, which we initialized earlier as `app.mongodb_encryption_client`. This encryption client is responsible for generating a DEK, which in this case will be encrypted by the envelope key. In our case, the encryption client is configured to use an envelope key from AWS KMS.

```python
# flaskapp/db_queries.py

@aws_credential_handler
def create_key(userId):
    data_key_id = \
        app.mongodb_encryption_client.create_data_key(kms_provider,
            master_key, key_alt_names=[userId])
    return data_key_id
```

We can then use this method when saving the user.

```python
# flaskapp/user.py

def save(self):
        dek_id = db_queries.create_key(self.username)
        result = app.mongodb[db_name].user.insert_one(
            {
                "username": self.username,
                "password_hash": self.password_hash,
                "dek_id": dek_id,
                "createdAt": datetime.now(),
            }
        )
        if result:
            self.id = result.inserted_id
            return True
        else:
            return False
```

Once signed up, the user can then log in, after which they can enter data via a form shown in the screenshot below. This data has a "name" and a "value", allowing the user to store arbitrary key-value pairs.

![demo application showing a form to add data](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/form_to_add_data_364735b756.png)

In the database, we'll store this data in a MongoDB collection called “data,” in documents structured like this:

```json
{
   "name": "shoe size",
   "value": "10",
   "username": "tom"
}
```

For the sake of this demonstration, we have chosen to encrypt the value and username fields from this document. Those fields will be encrypted using the DEK created on signup belonging to the logged in user.

```python
# flaskapp/db_queries.py

# Fields to encrypt, and the algorithm to encrypt them with
ENCRYPTED_FIELDS = {
    # Deterministic encryption for username, because we need to search on it
    "username": Algorithm.AEAD_AES_256_CBC_HMAC_SHA_512_Deterministic,
    # Random encryption for value, as we don't need to search on it
    "value": Algorithm.AEAD_AES_256_CBC_HMAC_SHA_512_Random,
}
```

The insert_data function then loops over the fields we want to encrypt and the algorithm we're using for each.

```python
# flaskapp/db_queries.py

def insert_data(document):
    document["username"] = current_user.username
    # Loop over the field names (and associated algorithm) we want to encrypt
    for field, algo in ENCRYPTED_FIELDS.items():
        # if the field exists in the document, encrypt it
        if document.get(field):
            document[field] = encrypt_field(document[field], algo)
    # Insert document (now with encrypted fields) to the data collection
    app.data_collection.insert_one(document)
```

If the specified fields exist in the document, this will call our encrypt_field function to perform the encryption using the specified algorithm.

```python
# flaskapp/db_queries.py

# Encrypt a single field with the given algorithm
@aws_credential_handler
def encrypt_field(field, algorithm):
    try:
        field = app.mongodb_encryption_client.encrypt(
            field,
            algorithm,
            key_alt_name=current_user.username,
        )
        return field
    except pymongo.errors.EncryptionError as ex:
        # Catch this error in case the DEK doesn't exist. Log a warning and 
        # re-raise the exception
        if "not all keys requested were satisfied" in ex._message:
            app.logger.warn(
                f"Encryption failed: could not find data encryption key for user: {current_user.username}"
            )
        raise ex
```

Once data is added, it will be shown in the web app:

![demo application showing the data added in the previous step](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/demo_application_showing_data_12dfb4df80.png)

Now let's see what happens if we delete the DEK. To do this, we can head over to the admin page. This admin page should only be provided to individuals that have a need to manage keys, and we have some choices:

![demo application admin page, showing the "delete data encryption key" button](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/demo_application_admin_page_829d227a7f.png)

We're going to use the "Delete data encryption key" option, which will remove the DEK, but leave all data entered by the user intact. After that, the application will no longer be able to retrieve the data that was stored via the form. When trying to retrieve the data for the logged in user, an error will be thrown

![demo application showing an error message when trying to retrieve data](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/error_message_e04de7ff46.png)

**Note**: After we do perform the data key deletion, the web application may still be able to decrypt and show the data for a short period of time before its cache expires — this takes a maximum of 60 seconds. 

But what is actually left in the database? To get a view of this, you can go back to the Admin page and choose "Fetch data for all users." In this view, we won't throw an exception if we can't decrypt the data. We'll just show exactly what we have stored in the database. Even though we haven't actually deleted the user's data, because the data encryption key no longer exists, all we can see now is cipher text for the encrypted fields "username" and "value".

![demo application showing raw cipher text instead of decrypted data](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/raw_cipher_text_8312deece3.png)

And here is the code we're using to fetch the data in this view. As you can see, we use very similar logic to the encrypt method shown earlier. We perform a find operation without any filters to retrieve all the data from our data collection. We'll then loop over our ENCRYPTED_FIELDS dictionary to see which fields need to be decrypted.

```python
# flaskapp/db_queries.py

def fetch_all_data_unencrypted(decrypt=False):
    results = list(app.data_collection.find())

    if decrypt:
        for field in ENCRYPTED_FIELDS.keys():
            for result in results:
                if result.get(field):
                    result[field], result["encryption_succeeded"] = decrypt_field(result[field])
    return results
```

The decrypt_field function is called for each field to be decrypted, but in this case we'll catch the error if we cannot successfully decrypt it due to a missing DEK.

```python
# flaskapp/db_queries.py

# Try to decrypt a field, returning a tuple of (value, status). This will be either (decrypted_value, True), or (raw_cipher_text, False) if we couldn't decrypt
def decrypt_field(field):
    try:
        # We don't need to pass the DEK or algorithm to decrypt a field
        field = app.mongodb_encryption_client.decrypt(field)
        return field, True
    # Catch this error in case the DEK doesn't exist.
    except pymongo.errors.EncryptionError as ex:
        if "not all keys requested were satisfied" in ex._message:
            app.logger.warn(
                "Decryption failed: could not find data encryption key to decrypt the record."
            )
            # If we can't decrypt due to missing DEK, return the "raw" value.
            return field, False
        raise ex
```

We can also use the `mongosh` shell to check directly in the database, just to prove that there's nothing there we can read. 

![mongosh](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/mongosh_ccc9e4e3a7.png)

At this point, savvy readers may be asking the question, "But what if we restore the database from a backup?" If we want to prevent this, we can use two separate database clusters in our application — one for storing data and one for storing DEKs (the "key vault"). This theory is applied in the sample application, which requires you to specify two MongoDB connection strings — one for data and one for the key vault. If we use separate clusters, it decouples the restoration of backups for application data and the key vault; restoring a backup on the data cluster won't restore any DEKs which have been deleted from the key vault cluster.

## Conclusion
In this blog post, we've demonstrated how MongoDB's Client-Side Field Level Encryption can be used to simplify the task of "forgetting" certain data. With a single "delete data key" operation, we can effectively forget data which may be stored across different databases, collections, backups, and logs. In a real production application, we may wish to delete all the user's data we can find, on top of removing their DEK. This "defense in depth" approach helps us to ensure that the data is really gone. By implementing crypto shredding, the impact is much smaller if a delete operation fails, or misses some data that should have been wiped.

You can find more details about MongoDB's Client-Side Field Level Encryption in our [documentation](https://www.mongodb.com/docs/manual/core/csfle/). If you have questions, feel free to make a post on our [community forums](https://www.mongodb.com/community/forums/). 

Learn how to make use of MongoDB's Client-Side Field Level Encryption to strengthen procedures for removing sensitive data.

Pierre Petersson

Content by Pierre Petersson

Building a Multi-Environment Continuous Delivery Pipeline for MongoDB Atlas

How to Set Up HashiCorp Vault KMIP Secrets Engine with MongoDB CSFLE or Queryable Encryption

Implementing Right to Erasure with CSFLE