HomeLearnHow-toIntegrate Azure Key Vault with MongoDB Client-Side Field Level Encryption

Integrate Azure Key Vault with MongoDB Client-Side Field Level Encryption

Updated: Aug 27, 2021 |

Published: Aug 27, 2021

  • MongoDB
  • Atlas
  • C#
  • ...

By Adrienne Tacke

Rate this article

When implementing MongoDB’s client-side field level encryption (CSFLE), you’ll find yourself making an important decision: Where do I store my customer master key? In another tutorial, I guided readers through the basics of CSFLE by using a locally-generated and stored master key. While this works for educational and local development purposes, it isn’t suitable for production! In this tutorial, we’ll see how to use Azure Key Vault to generate and securely store our master key.

#Prerequisites

#Quick Jump

Prepare Azure Infrastructure

Configure your Client Application to use Azure Key Vault and CSFLE

#Register App in Azure Active Directory

In order to establish a trust relationship between our application and the Microsoft identity platform, we first need to register it. 

  1. Sign in to the Azure portal.
  2. If you have access to multiple tenants, in the top menu, use the “Directory + subscription filter” to select the tenant in which you want to register an application.
  3. In the main search bar, search for and select “Azure Active Directory.”
  4. On the left-hand navigation menu, find the Manage section and select “App registrations,” then “+ New registration.”
  5. Enter a display name for your application. You can change the display name at any time and multiple app registrations can share the same name. The app registration's automatically generated Application (client) ID, not its display name, uniquely identifies your app within the identity platform.
  6. Specify who can use the application, sometimes called its sign-in audience. For this tutorial, I’ve selected “Accounts in this organizational directory only (Default Directory only - Single tenant).” This only allows users that are in my current tenant access to my application.
  7. Click “Register.” Once the initial app registration is complete, copy the Directory (tenant) ID and Application (client) ID as we’ll need them later on.
  8. Find the linked application under “Managed application in local directory” and click on it. https://lh4.googleusercontent.com/Xc6mphV7B5BnwojDmLEA2QnSG4xrqwfWOkOJvqG7uBbMNV_s6DTsKJFRyqaHryyziA5YBIZnc2aJ34DdCqalqpm8xCKdIrxAx2o2Pjx_whjiXM-rlmoV_l4NKZL3UmEpfP4W3ZVx=s0
  9. Once brought to the “Properties” page, also copy the “Object ID” as we’ll need this too.https://lh4.googleusercontent.com/2KgAuRdeJI7K3e6DBJXKGr7K8SNC6bikQmZJkXNpy7LupwvXfuJxTWHKCjtAJnTaUA7vqDYMQiJB1zjiOAbczVKmTujWP820Zey8ZYVZaMvKtk3qdsrOXlxiLL19VpUELcBDn2SP=s0

#Create a Client Secret

Once your application is registered, we’ll need to create a client secret for it. This will be required when authenticating to the Key Vault we’ll be creating soon.

  1. On the overview of your newly registered application, click on “Add a certificate or secret”:https://lh6.googleusercontent.com/GOPubpbAUfglFo6mcmG0TEd-ZXLkabsuY0S_p-3iMc1Fbn_Xv1QEArPZA-VoxZAVN3Mv_RzQK8o8DSNKPU9nRxmfNTwlxRQbRRESEUmtWjVFWDgSvvqhYzANy32hM9s1_TnATlOz=s0
  2. Under “Client secrets,” click “+ New client secret.”
  3. Enter a short description for this client secret and leave the default “Expires” setting of 6 months.
  4. Click “Ad." Once the client secret is created, be sure to copy the secret’s “Value” as we’ll need it later. It’s also worth mentioning that once you leave this page, the secret value is never displayed again, so be sure to record it at least once!

#Create an Azure Key Vault

Next up, an Azure Key Vault! We’ll create one so we can securely store our customer master key. We’ll be completing these steps via the Azure CLI, so open up your favorite terminal and follow along:

  1. Sign in to the Azure CLI using the az login command. Finish the authentication steps by following the steps displayed in your terminal.
  2. Create a resource group: 

    1az group create --name "YOUR-RESOURCE-GROUP-NAME" --location <YOUR-AZURE-REGION>
  3. Create a key vault: 

    1az keyvault create --name "YOUR-KEYVAULT-NAME" --resource-group "YOUR-RESOURCE-GROUP-NAME" --location <YOUR-AZURE-REGION>

#Create and Add a Key to Your Key Vault

With a key vault, we can now create our customer master key! This will be stored, managed, and secured by Azure Key Vault.

Create a key and add to our key vault:

1az keyvault key create --vault-name "YOUR-KEYVAULT-NAME" --name "YOUR-KEY-NAME" --protection software

The --protection parameter designates the key protection type. For now, we'll use the software type. Once the command completes, take note of your key’s "name" as we‘ll need it later!

#Grant Application Permissions to Key Vault

To enable our client application access to our key vault, some permissions need to be granted:

  1. Give your application the wrapKey and unwrapKey permissions to the keyvault. (For the --object-id parameter, paste in the Object ID of the application we registered earlier. This is the Object ID we copied in the last "Register App in Azure Active Directory" step.)

    1az keyvault set-policy --name "YOUR-KEYVAULT-NAME" --key-permissions wrapKey unwrapKey --object-id <YOUR-APP-OBJECT-ID>
  2. Upon success, you’ll receive a JSON object. Find and copy the value for the “vaultUri” key. For example, mine is https://csfle-mdb-demo-vault.vault.azure.net.

#Integrate Azure Key Vault into Your Client Application

Now that our cloud infrastructure is configured, we can start integrating it into our application. We’ll be referencing the sample repo from our prerequisites for these steps, but feel free to use the portions you need in an existing application.

  1. If you haven’t cloned the repo yet, do so now! 

    1git clone https://github.com/adriennetacke/mongodb-csfle-csharp-demo-azure.git
  2. Navigate to the root directory mongodb-csfle-csharp-demo-azure and open the EnvoyMedSys sample application in Visual Studio.
  3. In the Solution Explorer, find and open the launchSettings.json file (Properties > launchSettings.json).
  4. Here, you’ll see some scaffolding for some variables. Let’s quickly go over what those are:

    • MDB_ATLAS_URI: The connection string to your MongoDB Atlas cluster. This enables us to store our data encryption key, encrypted by Azure Key Vault.
    • AZURE_TENANT_ID: Identifies the organization of the Azure account.
    • AZURE_CLIENT_ID: Identifies the clientId to authenticate your registered application.
    • AZURE_CLIENT_SECRET: Used to authenticate your registered application.
    • AZURE_KEY_NAME: Name of the Customer Master Key stored in Azure Key Vault.
    • AZURE_KEYVAULT_ENDPOINT: URL of the Key Vault. E.g., yourVaultName.vault.azure.net.
  5. Replace all of the placeholders in the launchSettings.json file with your own information. Each variable corresponds to a value you were asked to copy and keep track of:

    • MDB_ATLAS_URI: Your Atlas URI.
    • AZURE_TENANT_IDDirectory (tenant) ID.
    • AZURE_CLIENT_IDApplication (client) ID.
    • AZURE_CLIENT_SECRET: Secret Value from our client secret.
    • AZURE_KEY_NAME: Key Name.
    • AZURE_KEYVAULT_ENDPOINT: Our Key Vault’s vaultUri.
  6. Save all your files!

Before we run the application, let’s go over what’s happening: When we run our main program, we set the connection to our Atlas cluster and our key vault’s collection namespace. We then instantiate two helper classes: a KmsKeyHelper and an AutoEncryptHelper. The KmsKeyHelper’s CreateKeyWithAzureKmsProvider() method is called to generate our encrypted data encryption key. This is then passed to the AutoEncryptHelper’s EncryptedWriteAndReadAsync() method to insert a sample document with encrypted fields and properly decrypt it when we need to fetch it. This is all in our Program.cs file:

Program.cs

1using System;
2using MongoDB.Driver;
3
4namespace EnvoyMedSys
5{
6 public enum KmsKeyLocation
7 {
8 Azure,
9 }
10
11 class Program
12 {
13 public static void Main(string[] args)
14 {
15 var connectionString = Environment.GetEnvironmentVariable("MDB_ATLAS_URI");
16 var keyVaultNamespace = CollectionNamespace.FromFullName("encryption.__keyVaultTemp");
17
18 var kmsKeyHelper = new KmsKeyHelper(
19 connectionString: connectionString,
20 keyVaultNamespace: keyVaultNamespace);
21 var autoEncryptHelper = new AutoEncryptHelper(
22 connectionString: connectionString,
23 keyVaultNamespace: keyVaultNamespace);
24
25 var kmsKeyIdBase64 = kmsKeyHelper.CreateKeyWithAzureKmsProvider().GetAwaiter().GetResult();
26
27 autoEncryptHelper.EncryptedWriteAndReadAsync(kmsKeyIdBase64, KmsKeyLocation.Azure).GetAwaiter().GetResult();
28
29 Console.ReadKey();
30 }
31 }
32}

Taking a look at the KmsKeyHelper class, there are a few important methods: the CreateKeyWithAzureKmsProvider() and GetClientEncryption() methods. I’ve opted to include comments in the code to make it easier to follow along:
KmsKeyHelper.csCreateKeyWithAzureKmsProvider()

1public async Task<string> CreateKeyWithAzureKmsProvider()
2{
3 var kmsProviders = new Dictionary<string, IReadOnlyDictionary<string, object>>();
4
5 // Pull Azure Key Vault settings from environment variables
6 var azureTenantId = Environment.GetEnvironmentVariable("AZURE_TENANT_ID");
7 var azureClientId = Environment.GetEnvironmentVariable("AZURE_CLIENT_ID");
8 var azureClientSecret = Environment.GetEnvironmentVariable("AZURE_CLIENT_SECRET");
9 var azureIdentityPlatformEndpoint = Environment.GetEnvironmentVariable("AZURE_IDENTIFY_PLATFORM_ENPDOINT"); // Optional, only needed if user is using a non-commercial Azure instance
10
11 // Configure our registered application settings
12 var azureKmsOptions = new Dictionary<string, object>
13 {
14 { "tenantId", azureTenantId },
15 { "clientId", azureClientId },
16 { "clientSecret", azureClientSecret },
17 };
18
19 if (azureIdentityPlatformEndpoint != null)
20 {
21 azureKmsOptions.Add("identityPlatformEndpoint", azureIdentityPlatformEndpoint);
22 }
23
24 // Specify remote key location; in this case, Azure
25 kmsProviders.Add("azure", azureKmsOptions);
26
27 // Constructs our client encryption settings which
28 // specify which key vault client, key vault namespace,
29 // and KMS providers to use.
30 var clientEncryption = GetClientEncryption(kmsProviders);
31
32 // Set KMS Provider Settings
33 // Client uses these settings to discover the master key
34 var azureKeyName = Environment.GetEnvironmentVariable("AZURE_KEY_NAME");
35 var azureKeyVaultEndpoint = Environment.GetEnvironmentVariable("AZURE_KEYVAULT_ENDPOINT"); // typically <azureKeyName>.vault.azure.net
36 var azureKeyVersion = Environment.GetEnvironmentVariable("AZURE_KEY_VERSION"); // Optional
37 var dataKeyOptions = new DataKeyOptions(
38 masterKey: new BsonDocument
39 {
40 { "keyName", azureKeyName },
41 { "keyVaultEndpoint", azureKeyVaultEndpoint },
42 { "keyVersion", () => azureKeyVersion, azureKeyVersion != null }
43 });
44
45 // Create Data Encryption Key
46 var dataKeyId = clientEncryption.CreateDataKey("azure", dataKeyOptions, CancellationToken.None);
47 Console.WriteLine($"Azure DataKeyId [UUID]: {dataKeyId}");
48
49 var dataKeyIdBase64 = Convert.ToBase64String(GuidConverter.ToBytes(dataKeyId, GuidRepresentation.Standard));
50 Console.WriteLine($"Azure DataKeyId [base64]: {dataKeyIdBase64}");
51
52 // Optional validation; checks that key was created successfully
53 await ValidateKeyAsync(dataKeyId);
54
55 return dataKeyIdBase64;
56}

KmsKeyHelper.cs / GetClientEncryption()

1private ClientEncryption GetClientEncryption( Dictionary<string, IReadOnlyDictionary<string, object>> kmsProviders)
2{
3 // Construct a MongoClient using our Atlas connection string
4 var keyVaultClient = new MongoClient(_mdbConnectionString);
5
6 // Set MongoClient, key vault namespace, and Azure as KMS provider
7 var clientEncryptionOptions = new ClientEncryptionOptions(
8 keyVaultClient: keyVaultClient,
9 keyVaultNamespace: _keyVaultNamespace,
10 kmsProviders: kmsProviders);
11
12 return new ClientEncryption(clientEncryptionOptions);
13}

With our Azure Key Vault connected and data encryption key encrypted, we’re ready to insert some data into our Atlas cluster! This is where the AutoEncryptHelper class comes in. The important method to note here is the EncryptedReadAndWrite() method:

AutoEncryptHelper.cs / EncryptedReadAndWrite()

1public async Task EncryptedWriteAndReadAsync(string keyIdBase64, KmsKeyLocation kmsKeyLocation)
2{
3 // Construct a JSON Schema
4 var schema = JsonSchemaCreator.CreateJsonSchema(keyIdBase64);
5
6 // Construct an auto-encrypting client
7 var autoEncryptingClient = CreateAutoEncryptingClient(
8 kmsKeyLocation,
9 _keyVaultNamespace,
10 schema);
11
12 // Set our working database and collection to medicalRecords.patientData
13 var collection = autoEncryptingClient
14 .GetDatabase(_medicalRecordsNamespace.DatabaseNamespace.DatabaseName)
15 .GetCollection<BsonDocument>(_medicalRecordsNamespace.CollectionName);
16
17 var ssnQuery = Builders<BsonDocument>.Filter.Eq("ssn", __sampleSsnValue);
18
19 // Upsert (update if found, otherwise create it) a document into the collection
20 var medicalRecordUpdateResult = await collection
21 .UpdateOneAsync(ssnQuery, new BsonDocument("$set", __sampleDocFields), new UpdateOptions() { IsUpsert = true });
22
23 if (!medicalRecordUpdateResult.UpsertedId.IsBsonNull)
24 {
25 Console.WriteLine("Successfully upserted the sample document!");
26 }
27
28 // Query by SSN field with auto-encrypting client
29 var result = await collection.Find(ssnQuery).SingleAsync();
30
31 // Proper result in console should show decrypted, human-readable document
32 Console.WriteLine($"Encrypted client query by the SSN (deterministically-encrypted) field:\n {result}\n");
33}

Now that we know what’s going on, run your application!

#The Results: What You Get After Integrating Azure Key Vault with MongoDB CSFLE

If all goes well, your console will print out two DataKeyIds (UUID and base64) and a document that resembles the following: 

Sample Result Document (using my information)

1{
2 _id:UUID('ab382f3e-bc79-4086-8418-836a877efff3'),
3keyMaterial:Binary('tvehP03XhUsztKr69lxlaGjiPhsNPjy6xLhNOLTpe4pYMeGjMIwvvZkzrwLRCHdaB3vqi9KKe6/P5xvjwlVHacQ1z9oFIwFbp9nk...', 0),
4 creationDate:2021-08-24T05:01:34.369+00:00,
5 updateDate:2021-08-24T05:01:34.369+00:00,
6 status:0,
7 masterKey:Object,
8 provider:"azure",
9 keyVaultEndpoint:"csfle-mdb-demo-vault.vault.azure.net",
10 keyName:"MainKey"
11}

Here’s what my console output looks like, for reference:

Screenshot of console output showing two Azure DatakeyIds and a non-formatted document

Seeing this is great news! A lot of things have just happened, and all of them are good:

  • Our application properly authenticated to our Azure Key Vault.
  • A properly generated data encryption key was created by our client application.
  • The data encryption key was properly encrypted by our customer master key that’s securely stored in Azure Key Vault.
  • The encrypted data encryption key was returned to our application and stored in our MongoDB Atlas cluster.

Here’s the same process in a workflow:

Workflow diagram of how a data encryption key is set up with a KMS Provider. Client application creates an unecrypted data encryption key which requests encryption from Azure Key Vault (the KMS provider). using the Customer Master Key, Azure Key Vault encrypts the data encryption key, sends it back to the client application which then stores the encrypted key in MongoDB's key vault collection.

After a few more moments, and upon success, you’ll see a “Successfully upserted the sample document!” message, followed by the properly decrypted results of a test query. Again, here’s my console output for reference:

Screenshot of console output showing a "Successfully upserted the sample document message!" followed by a human-readable document of a sample patient's data, including properly decrypted SSN, policyNumber fields.

This means our sample document was properly encrypted, inserted into our patientData collection, queried with our auto-encrypting client by SSN, and had all relevant fields correctly decrypted before returning them to our console. How neat! 

And just because I’m a little paranoid, we can double-check that our data has actually been encrypted. If you log into your Atlas cluster and navigate to the patientData collection, you’ll see that our documents’ sensitive fields are all illegible:

Screenshot of encrypted document in Atlas cluster. Shows SSN, bloodType, polucyNumber, medicalRecord fields as asterisks and other fields as legible.

#Let's Summarize

That wasn’t so bad, right? Let's see what we've accomplished! This tutorial walked you through:

  • Registering an App in Azure Active Directory.
  • Creating a Client Secret.
  • Creating an Azure Key Vault.
  • Creating and Adding a Key to your Key Vault.
  • Granting Application Permissions to Key Vault.
  • Integrating Azure Key Vault into Your Client Application.
  • The Results: What You Get After Integrating Azure Key Vault with MongoDB CSFLE.

By using a remote key management system like Azure Key Vault, you gain access to many benefits over using a local filesystem. The most important of these is the secure storage of the key, reduced risk of access permission issues, and easier portability!

For more information, check out this helpful list of resources I used while preparing this tutorial:

And if you have any questions or need some additional help, be sure to check us out on the MongoDB Community Forums and start a topic!

A whole community of MongoDB engineers (including the DevRel team) and fellow developers are sure to help!

Rate this article

Related

© 2021 MongoDB, Inc.

About

  • Careers
  • Legal Notices
  • Privacy Notices
  • Security Information
  • Trust Center
© 2021 MongoDB, Inc.