How to Use MongoDB Client-Side Field Level Encryption (CSFLE) with C#
Rate this code example
Client-side field level encryption (CSFLE) provides an additional layer of security to your most sensitive data. Using a supported MongoDB driver, CSFLE encrypts certain fields that you specify, ensuring they are never transmitted unencrypted, nor seen unencrypted by the MongoDB server.
This also means that it's nearly impossible to obtain sensitive information from the database server. Without access to a specific key, data cannot be decrypted and exposed, rendering the intercepting data from the client fruitless. Reading data directly from disk, even with DBA or root credentials, will also be impossible as the data is stored in an encrypted state.
Key applications that showcase the power of client-side field level encryption are those in the medical field. If you quickly think back to the last time you visited a clinic, you already have an effective use case for an application that requires a mix of encrypted and non-encrypted fields. When you check into a clinic, the person may need to search for you by name or insurance provider. These are common data fields that are usually left non-encrypted. Then, there are more obvious pieces of information that require encryption: things like a Social Security number, medical records, or your insurance policy number. For these data fields, encryption is necessary.
In it, you'll:
Prepare a .NET Core console application
Generate secure, random keys needed for CSFLE
Configure CSFLE on the MongoClient
See CSFLE in action
💡️ This can be an intimidating tutorial, so don't hesitate to take as many breaks as you need; in fact, complete the steps over a few days! I've tried my best to ensure each step completed acts as a natural save point for the duration of the entire tutorial. :)
Let's do this step by step!
- A MongoDB Atlas cluster running MongoDB 4.2 (or later) OR MongoDB 4.2 Enterprise Server (or later)—required for automatic encryption
- MongoDB .NET Driver 2.12.0-beta (or later)
- File system permissions (to start the mongocryptd process, if running locally)
Let's start by scaffolding our console application. Open Visual Studio (I'm using Visual Studio 2019 Community Edition) and create a new project. When selecting a template, choose the "Console App (.NET Core)" option and follow the prompts to name your project.
Once the project template loads, we'll need to install one of our dependencies. In your Package Manager Console, use the following command to install the MongoDB Driver:
💡️ If your Package Manager Console is not visible in your IDE, you can get to it via View > Other Windows > Package Manager Console in the File Menu.
The next dependency you'll need to install is mongocryptd, which is an application that is provided as part of MongoDB Enterprise and is needed for automatic field level encryption. Follow the instructions to install mongocryptd on your machine. In a production environment, it's recommended to run mongocryptd as a service at startup on your VM or container.
Now that our base project and dependencies are set, we can move onto creating and configuring our different encryption keys.
MongoDB client-side field level encryption uses an encryption strategy called envelope encryption. This strategy uses two different kinds of keys.
The first key is called a data encryption key, which is used to encrypt/decrypt the data you'll be storing in MongoDB. The other key is called a master key and is used to encrypt the data encryption key. This is the top-level plaintext key that will always be required and is the key we are going to generate in the next step.
🚨️ Before we proceed, it's important to note that this tutorial will
demonstrate the generation of a master key file stored as plaintext in
the root of our application. This is okay for development and
educational purposes, such as this tutorial. However, this should
NOT be done in a production environment!
Why? In this scenario, anyone that obtains a copy of the disk or a VM
snapshot of the app server hosting our application would also have
access to this key file, making it possible to access the application's
data.
Instead, you should configure a master key in a Key Management
System
such as Azure Key Vault or AWS KMS for production.
Keep this in mind and watch for another post that shows how to implement
CSFLE with Azure Key Vault!
In this step, we generate a 96-byte, locally-managed master key. Then, we save it to a local file called
master-key.txt. We'll be doing a few more things with keys, so create a separate class called KmsKeyHelper.cs. Then, add the following code to it:So, what's happening here? Let's break it down, line by line:
First, we declare and set a private variable called
__localMasterKeyPath. This holds the path to where we save our master key.Next, we create a
GenerateLocalMasterKey() method. In this method, we use .NET's Cryptography services to create an instance of a RandomNumberGenerator. Using this RandomNumberGenerator, we generate a cryptographically strong, 96-byte key. After converting it to a Base64 representation, we save the key to the master-key.txt file.Great! We now have a way to generate a local master key. Let's modify the main program to use it. In the
Program.cs file, add the following code:In the
Main method, we create an instance of our KmsKeyHelper, then call our GenerateLocalMasterKey() method. Pretty straightforward!Save all files, then run your program. If all is successful, you'll see a console pop up and the Base64 representation of your newly generated master key printed in the console. You'll also see a new
master-key.txt file appear in your solution explorer.Now that we have a master key, we can move onto creating a data encryption key.
The next key we need to generate is a data encryption key. This is the key the MongoDB driver stores in a key vault collection, and it's used for automatic encryption and decryption.
Automatic encryption requires MongoDB Enterprise 4.2 or a MongoDB 4.2 Atlas cluster. However, automatic decryption is supported for all users. See how to configure automatic decryption without automatic encryption.
Let's add a few more lines of code to the
Program.cs file:So, what's changed? First, we added an additional import (
MongoDB.Driver). Next, we declared a connectionString and a keyVaultNamespace variable.For the key vault namespace, MongoDB will automatically create the database
encryption and collection __keyVault if it does not currently exist. Both the database and collection names were purely my preference. You can choose to name them something else if you'd like!Next, we modified the
KmsKeyHelper instantiation to accept two parameters: the connection string and key vault namespace we previously declared. Don't worry, we'll be changing our KmsKeyHelper.cs file to match this soon.Finally, we declare a
kmsKeyIdBase64 variable and set it to a new method we'll create soon: CreateKeyWithLocalKmsProvider();. This will hold our data encryption key.In our code, we set our MongoDB URI by pulling from environment variables. This is far safer than pasting a connection string directly into our code and is scalable in a variety of automated deployment scenarios.
For our purposes, we'll create a
launchSettings.json file.💡️ Don't commit the
launchSettings.json file to a public repo! In
fact, add it to your .gitignore file now, if you have one or plan to
share this application. Otherwise, you'll expose your MongoDB URI to the
world!Right-click on your project and select "Properties" in the context menu.
The project properties will open to the "Debug" section. In the "Environment variables:" area, add a variable called
MDB_URI, followed by the connection URI:
What value do you set to your
MDB_URI environment variable?- MongoDB Atlas: If using a MongoDB Atlas cluster, paste in your Atlas URI.
- Local: If running a local MongoDB instance and haven't changed any default settings, you can use the default connection string:
mongodb://localhost:27017.
Once your
MDB_URI is added, save the project properties. You'll see that a launchSettings.json file will be automatically generated for you! Now, any Environment.GetEnvironmentVariable() calls will pull from this file.With these changes, we now have to modify and add a few more methods to the
KmsKeyHelper class. Let's do that now.First, add these additional imports:
Next, declare two private variables and create a constructor that accepts both a connection string and key vault namespace. We'll need this information to create our data encryption key; this also makes it easier to extend and integrate with a remote KMS later on.
After the GenerateLocalMasterKey() method, add the following new methods. Don't worry, we'll go over each one:
This method is the one we call from the main program. It's here that we generate our data encryption key. Lines 6-7 read the local master key from our
master-key.txt file and convert it to a byte array.Lines 11-16 set the KMS provider settings the client needs in order to discover the master key. As you can see, we add the local provider and the matching local master key we've just retrieved.
With these KMS provider settings, we construct additional client encryption settings. We do this in a separate method called
GetClientEncryption(). Once created, we finally generate an encrypted key.As an extra measure, we call a third new method
ValidateKey(), just to make sure the data encryption key was created. After these steps, and if successful, the CreateKeyWithLocalKmsProvider() method returns our data key id encoded in Base64 format.After the CreateKeyWithLocalKmsProvider() method, add the following method:
Within the
CreateKeyWithLocalKmsProvider() method, we call GetClientEncryption() (the method we just added) to construct our client encryption settings. These include which key vault client, key vault namespace, and KMS providers to use.In this method, we construct a MongoClient using the connection string, then set it as a key vault client. We also use the key vault namespace that was passed in and the local KMS providers we previously constructed. These client encryption options are then returned.
Last but not least, after GetClientEncryption(), add the final method:
Though optional, this method conveniently checks that the data encryption key was created correctly. It does this by constructing a MongoClient using the specified connection string, then queries the database for the data encryption key. If it was successfully created, the data encryption key would have been inserted as a document into your replica set and will be retrieved in the query.
With these changes, we're ready to generate our data encryption key. Make sure to save all files, then run your program. If all goes well, your console will print out two DataKeyIds (UUID and base64) as well as a document that resembles the following:
For reference, here's what my console output looks like:
If you want to be extra sure, you can also check your cluster to see that your data encryption key is stored as a document in the newly created encryption database and __keyVault collection. Since I'm connecting with my Atlas cluster, here's what it looks like there:
Sweet! Now that we have generated a data encryption key, which has been encrypted itself with our local master key, the next step is to specify which fields in our application should be encrypted.
In order for automatic client-side encryption and decryption to work, a JSON schema needs to be defined that specifies which fields to encrypt, which encryption algorithms to use, and the BSON Type of each field.
Using our medical application as an example, let's plan on encrypting the following fields:
| Field name | Encryption algorithms | BSON Type |
|---|---|---|
| SSN (Social Security Number) | Deterministic | Int |
| Blood Type | Random | String |
| Medical Records | Random | Array |
| Insurance: Policy Number | Deterministic | Int (embedded inside insurance object) |
To make this a bit easier, and to separate this functionality from the rest of the application, create another class named
JsonSchemaCreator.cs. In it, add the following code:As before, let's step through each line:
First, we create two static variables to hold our encryption types. We use
Deterministic encryption for fields that are queryable and have high cardinality. We use Random encryption for fields we don't plan to query, have low cardinality, or are array fields.Next, we create a
CreateEncryptMetadata() helper method. This will return a BsonDocument that contains our converted data key. We'll use this key in the CreateJsonSchema() method.Lines 19-32 make up another helper method called
CreateEncryptedField(). This generates the proper BsonDocument needed to define our encrypted fields. It will output a BsonDocument that resembles the following:Finally, the
CreateJsonSchema() method. Here, we generate the full schema our application will use to know which fields to encrypt and decrypt. This method also returns a BsonDocument.A few things to note about this schema:
Placing the
encryptMetadata key at the root of our schema allows us to encrypt all fields with a single data key. It's here you see the call to our CreateEncryptMetadata() helper method.Within the
properties key go all the fields we wish to encrypt. So, for our ssn, bloodType, medicalRecords, and insurance.policyNumber fields, we generate the respective BsonDocument specifications they need using our CreateEncryptedField() helper method.With our encrypted fields defined and the necessary encryption keys generated, we can now move onto enabling client-side field level encryption in our MongoDB client!
☕️ Don't forget to take a break! This is a lot of information to take
in, so don't rush. Be sure to save all your files, then grab a coffee,
stretch, and step away from the computer. This tutorial will be here
waiting when you're ready. :)
A CSFLE-enabled
MongoClient is not that much different from a standard client. To create an auto-encrypting client, we instantiate it with some additional auto-encryption options.As before, let's create a separate class to hold this functionality. Create a file called
AutoEncryptHelper.cs and add the following code (note that since this is a bit longer than the other code snippets, I've opted to add inline comments to explain what's happening rather than waiting until after the code block):Alright, we're almost done. Don't forget to save what you have so far! In our next (and final) step, we can finally try out client-side field level encryption with some queries!
🌟 Know what show this patient is from? Let me know your nerd cred (and
let's be friends, fellow fan!) in a
tweet!
Remember the sample data we've prepared? Let's put that to good use! To test out an encrypted write and read of this data, let's add another method to the
AutoEncryptHelper class. Right after the constructor, add the following method:What's happening here? First, we use the
JsonSchemaCreator class to construct our schema. Then, we create an auto-encrypting client using the CreateAutoEncryptingClient() method. Next, lines 14-16 set the working database and collection we'll be interacting with. Finally, we upsert a medical record using our sample data, then retrieve it with the auto-encrypting client.Prior to inserting this new patient record, the CSFLE-enabled client automatically encrypts the appropriate fields as established in our JSON schema.
If you like diagrams, here's what's happening:
When retrieving the patient's data, it is decrypted by the client. The nicest part about enabling CSFLE in your application is that the queries don't change, meaning the driver methods you're already familiar with can still be used.
For the diagram people:
To see this in action, we just have to modify the main program slightly so that we can call the
EncryptedWriteAndReadAsync() method.Back in the
Program.cs file, add the following code:Alright, this is it! Save your files and then run your program. After a short wait, you should see the following console output:
It works! The console output you see has been decrypted correctly by our CSFLE-enabled MongoClient. We can also verify that this patient record has been properly saved to our database. Logging into my Atlas cluster, I see Takeshi's patient record stored securely, with the specified fields encrypted:
To see how these queries perform when using a non-encrypting client, let's add one more method to the
AutoEncryptHelper class. Right after the EncryptedWriteAndReadAsync() method, add the following:Here, we instantiate a standard MongoClient with no auto-encryption settings. Notice that we query by the non-encrypted
name field; this is because we can't query on encrypted fields using a MongoClient without CSFLE enabled.Finally, add a call to this new method in the
Program.cs file:Save all your files, then run your program again. You'll see your last query returns an encrypted patient record, as expected. Since we are using a non CSFLE-enabled MongoClient, no decryption happens, leaving only the non-encrypted fields legible to us:
Cheers! You've made it this far!
This tutorial walked you through:
- Creating a .NET Core console application.
- Installing dependencies needed to enable client-side field level encryption for your .NET core app.
- Creating a local master key.
- Creating a data encryption key.
- Constructing a JSON Schema to establish which fields to encrypt.
- Configuring a CSFLE-enabled MongoClient.
- Performing an encrypted read and write of a sample patient record.
- Performing a read using a non-CSFLE-enabled MongoClient to see the difference in the retrieved data.
With this knowledge of client-side field level encryption, you should be able to better secure applications and understand how it works!
I hope this tutorial made client-side field level encryption simpler to
integrate into your .NET application! If you have any further questions
or are stuck on something, head over to the MongoDB Community
Forums and start a
topic. A whole community of MongoDB engineers (including the DevRel
team) and fellow developers are sure to help!
In case you want to learn a bit more, here are the resources that were
crucial to helping me write this tutorial:
Rate this code example