How to Use MongoDB Client-Side Field Level Encryption (CSFLE) with C#
This also means that it's nearly impossible to obtain sensitive information from the database server. Without access to a specific key, data cannot be decrypted and exposed, rendering the intercepting data from the client fruitless. Reading data directly from disk, even with DBA or root credentials, will also be impossible as the data is stored in an encrypted state.
Key applications that showcase the power of client-side field level encryption are those in the medical field. If you quickly think back to the last time you visited a clinic, you already have an effective use case for an application that requires a mix of encrypted and non-encrypted fields. When you check into a clinic, the person may need to search for you by name or insurance provider. These are common data fields that are usually left non-encrypted. Then, there are more obvious pieces of information that require encryption: things like a Social Security number, medical records, or your insurance policy number. For these data fields, encryption is necessary.
In it, you'll:
Prepare a .NET Core console application
Generate secure, random keys needed for CSFLE
Configure CSFLE on the MongoClient
See CSFLE in action
💡️ This can be an intimidating tutorial, so don't hesitate to take as many breaks as you need; in fact, complete the steps over a few days! I've tried my best to ensure each step completed acts as a natural save point for the duration of the entire tutorial. :)
Let's do this step by step!
- File system permissions (to start the mongocryptd process, if running locally)
Let's start by scaffolding our console application. Open Visual Studio (I'm using Visual Studio 2019 Community Edition) and create a new project. When selecting a template, choose the "Console App (.NET Core)" option and follow the prompts to name your project.
Once the project template loads, we'll need to install one of our dependencies. In your Package Manager Console, use the following command to install the MongoDB Driver:
💡️ If your Package Manager Console is not visible in your IDE, you can get to it via View > Other Windows > Package Manager Console in the File Menu.
The next dependency you'll need to install is
, which is an application that is provided as part of MongoDB Enterprise and is needed for automatic field level encryption. Follow the
install mongocryptd on your machine. In a production environment, it's recommended to run mongocryptd as a service at startup on your VM or container.
Now that our base project and dependencies are set, we can move onto creating and configuring our different encryption keys.
The first key is called a data encryption key, which is used to encrypt/decrypt the data you'll be storing in MongoDB. The other key is called a master key and is used to encrypt the data encryption key. This is the top-level plaintext key that will always be required and is the key we are going to generate in the next step.
🚨️ Before we proceed, it's important to note that this tutorial will demonstrate the generation of a master key file stored as plaintext in the root of our application. This is okay for development and educational purposes, such as this tutorial. However, this should NOT be done in a production environment!
Why? In this scenario, anyone that obtains a copy of the disk or a VM snapshot of the app server hosting our application would also have access to this key file, making it possible to access the application's data.
Keep this in mind and watch for another post that shows how to implement CSFLE with Azure Key Vault!
In this step, we generate a 96-byte, locally-managed master key. Then, we save it to a local file called
master-key.txt. We'll be doing a few more things with keys, so create a separate class called
KmsKeyHelper.cs. Then, add the following code to it:
So, what's happening here? Let's break it down, line by line:
First, we declare and set a private variable called
__localMasterKeyPath. This holds the path to where we save our master key.
Next, we create a
GenerateLocalMasterKey()method. In this method, we use .NET's
services to create an instance of a
RandomNumberGenerator. Using this
RandomNumberGenerator, we generate a cryptographically strong, 96-byte key. After converting it to a Base64 representation, we save the key to the
Great! We now have a way to generate a local master key. Let's modify the main program to use it. In the
Program.csfile, add the following code:
Mainmethod, we create an instance of our
KmsKeyHelper, then call our
GenerateLocalMasterKey()method. Pretty straightforward!
Save all files, then run your program. If all is successful, you'll see a console pop up and the Base64 representation of your newly generated master key printed in the console. You'll also see a new
master-key.txtfile appear in your solution explorer.
Now that we have a master key, we can move onto creating a data encryption key.
The next key we need to generate is a data encryption key. This is the key the MongoDB driver stores in a key vault collection, and it's used for automatic encryption and decryption.
Let's add a few more lines of code to the
So, what's changed? First, we added an additional import (
MongoDB.Driver). Next, we declared a
For the key vault namespace, MongoDB will automatically create the database
__keyVaultif it does not currently exist. Both the database and collection names were purely my preference. You can choose to name them something else if you'd like!
Next, we modified the
KmsKeyHelperinstantiation to accept two parameters: the connection string and key vault namespace we previously declared. Don't worry, we'll be changing our
KmsKeyHelper.csfile to match this soon.
Finally, we declare a
kmsKeyIdBase64variable and set it to a new method we'll create soon:
CreateKeyWithLocalKmsProvider();. This will hold our data encryption key.
In our code, we set our MongoDB URI by pulling from environment variables. This is far safer than pasting a connection string directly into our code and is scalable in a variety of automated deployment scenarios.
For our purposes, we'll create a
💡️ Don't commit the
launchSettings.jsonfile to a public repo! In fact, add it to your
.gitignorefile now, if you have one or plan to share this application. Otherwise, you'll expose your MongoDB URI to the world!
Right-click on your project and select "Properties" in the context menu.
The project properties will open to the "Debug" section. In the "Environment variables:" area, add a variable called
MDB_URI, followed by the connection URI:
What value do you set to your
- MongoDB Atlas: If using a MongoDB Atlas cluster, paste in your Atlas URI.
- Local: If running a local MongoDB instance and haven't changed any default settings, you can use the default connection string:
MDB_URIis added, save the project properties. You'll see that a
launchSettings.jsonfile will be automatically generated for you! Now, any
Environment.GetEnvironmentVariable()calls will pull from this file.
With these changes, we now have to modify and add a few more methods to the
KmsKeyHelperclass. Let's do that now.
First, add these additional imports:
Next, declare two private variables and create a constructor that accepts both a connection string and key vault namespace. We'll need this information to create our data encryption key; this also makes it easier to extend and integrate with a remote KMS later on.
After the GenerateLocalMasterKey() method, add the following new methods. Don't worry, we'll go over each one:
This method is the one we call from the main program. It's here that we generate our data encryption key. Lines 6-7 read the local master key from our
master-key.txtfile and convert it to a byte array.
Lines 11-16 set the KMS provider settings the client needs in order to discover the master key. As you can see, we add the local provider and the matching local master key we've just retrieved.
With these KMS provider settings, we construct additional client encryption settings. We do this in a separate method called
GetClientEncryption(). Once created, we finally generate an encrypted key.
As an extra measure, we call a third new method
ValidateKey(), just to make sure the data encryption key was created. After these steps, and if successful, the
CreateKeyWithLocalKmsProvider()method returns our data key id encoded in Base64 format.
After the CreateKeyWithLocalKmsProvider() method, add the following method:
CreateKeyWithLocalKmsProvider()method, we call
GetClientEncryption()(the method we just added) to construct our client encryption settings. These include which key vault client, key vault namespace, and KMS providers to use.
In this method, we construct a MongoClient using the connection string, then set it as a key vault client. We also use the key vault namespace that was passed in and the local KMS providers we previously constructed. These client encryption options are then returned.
Last but not least, after GetClientEncryption(), add the final method:
Though optional, this method conveniently checks that the data encryption key was created correctly. It does this by constructing a MongoClient using the specified connection string, then queries the database for the data encryption key. If it was successfully created, the data encryption key would have been inserted as a document into your replica set and will be retrieved in the query.
With these changes, we're ready to generate our data encryption key. Make sure to save all files, then run your program. If all goes well, your console will print out two DataKeyIds (UUID and base64) as well as a document that resembles the following:
For reference, here's what my console output looks like:
If you want to be extra sure, you can also check your cluster to see that your data encryption key is stored as a document in the newly created encryption database and __keyVault collection. Since I'm connecting with my Atlas cluster, here's what it looks like there:
Sweet! Now that we have generated a data encryption key, which has been encrypted itself with our local master key, the next step is to specify which fields in our application should be encrypted.
In order for automatic client-side encryption and decryption to work, a JSON schema needs to be defined that specifies which fields to encrypt, which encryption algorithms to use, and the BSON Type of each field.
Using our medical application as an example, let's plan on encrypting the following fields:
To make this a bit easier, and to separate this functionality from the rest of the application, create another class named
JsonSchemaCreator.cs. In it, add the following code:
As before, let's step through each line:
First, we create two static variables to hold our encryption types. We use
Deterministicencryption for fields that are queryable and have high cardinality. We use
Randomencryption for fields we don't plan to query, have low cardinality, or are array fields.
Next, we create a
CreateEncryptMetadata()helper method. This will return a
BsonDocumentthat contains our converted data key. We'll use this key in the
Lines 19-32 make up another helper method called
CreateEncryptedField(). This generates the proper
BsonDocumentneeded to define our encrypted fields. It will output a
BsonDocumentthat resembles the following:
CreateJsonSchema()method. Here, we generate the full schema our application will use to know which fields to encrypt and decrypt. This method also returns a
A few things to note about this schema:
encryptMetadatakey at the root of our schema allows us to encrypt all fields with a single data key. It's here you see the call to our
propertieskey go all the fields we wish to encrypt. So, for our
insurance.policyNumberfields, we generate the respective
BsonDocumentspecifications they need using our
With our encrypted fields defined and the necessary encryption keys generated, we can now move onto enabling client-side field level encryption in our MongoDB client!
☕️ Don't forget to take a break! This is a lot of information to take in, so don't rush. Be sure to save all your files, then grab a coffee, stretch, and step away from the computer. This tutorial will be here waiting when you're ready. :)
MongoClientis not that much different from a standard client. To create an auto-encrypting client, we instantiate it with some additional auto-encryption options.
As before, let's create a separate class to hold this functionality. Create a file called
AutoEncryptHelper.csand add the following code (note that since this is a bit longer than the other code snippets, I've opted to add inline comments to explain what's happening rather than waiting until after the code block):
Alright, we're almost done. Don't forget to save what you have so far! In our next (and final) step, we can finally try out client-side field level encryption with some queries!
Remember the sample data we've prepared? Let's put that to good use! To test out an encrypted write and read of this data, let's add another method to the
AutoEncryptHelperclass. Right after the constructor, add the following method:
What's happening here? First, we use the
JsonSchemaCreatorclass to construct our schema. Then, we create an auto-encrypting client using the
CreateAutoEncryptingClient()method. Next, lines 14-16 set the working database and collection we'll be interacting with. Finally, we
a medical record using our sample data, then retrieve it with the auto-encrypting client.
Prior to inserting this new patient record, the CSFLE-enabled client automatically encrypts the appropriate fields as established in our JSON schema.
If you like diagrams, here's what's happening:
When retrieving the patient's data, it is decrypted by the client. The nicest part about enabling CSFLE in your application is that the queries don't change, meaning the driver methods you're already familiar with can still be used.
For the diagram people:
To see this in action, we just have to modify the main program slightly so that we can call the
Back in the
Program.csfile, add the following code:
Alright, this is it! Save your files and then run your program. After a short wait, you should see the following console output:
It works! The console output you see has been decrypted correctly by our CSFLE-enabled MongoClient. We can also verify that this patient record has been properly saved to our database. Logging into my Atlas cluster, I see Takeshi's patient record stored securely, with the specified fields encrypted:
To see how these queries perform when using a non-encrypting client, let's add one more method to the
AutoEncryptHelperclass. Right after the
EncryptedWriteAndReadAsync()method, add the following:
Here, we instantiate a standard MongoClient with no auto-encryption settings. Notice that we query by the non-encrypted
namefield; this is because we can't query on encrypted fields using a MongoClient without CSFLE enabled.
Finally, add a call to this new method in the
Save all your files, then run your program again. You'll see your last query returns an encrypted patient record, as expected. Since we are using a non CSFLE-enabled MongoClient, no decryption happens, leaving only the non-encrypted fields legible to us:
Cheers! You've made it this far!
This tutorial walked you through:
- Creating a .NET Core console application.
- Installing dependencies needed to enable client-side field level encryption for your .NET core app.
- Creating a local master key.
- Creating a data encryption key.
- Constructing a JSON Schema to establish which fields to encrypt.
- Configuring a CSFLE-enabled MongoClient.
- Performing an encrypted read and write of a sample patient record.
- Performing a read using a non-CSFLE-enabled MongoClient to see the difference in the retrieved data.
With this knowledge of client-side field level encryption, you should be able to better secure applications and understand how it works!
I hope this tutorial made client-side field level encryption simpler to integrate into your .NET application! If you have any further questions or are stuck on something, head over to the
Forums and start a topic. A whole community of MongoDB engineers (including the DevRel team) and fellow developers are sure to help!
In case you want to learn a bit more, here are the resources that were crucial to helping me write this tutorial: