Secure MongoDB with X.509 Authentication

Disclaimer - the following article is intended for a test environment.

Overview

In this tutorial, I will be describing the detailed process of setting up X.509 based authentication, both for cluster inter-member authentication as well as for client authentication, using a local CA (Certificate Authority).

X.509 is one of the multiple authentication mechanisms supported by MongoDB. An X.509 certificate is a digital certificate that uses the widely accepted international X.509 public key infrastructure (PKI) standard to verify that a public key, presented by a client or another member of the cluster, belongs to that said client or member. One of the main benefits compared to conventional password based authentication is it’s more secure in a sense that each machine would need a dedicated key to join the cluster. So stealing an existing key from another machine isn't going to be very helpful for those with an evil agenda.

In MongoDB, we need to understand the distinction between member authentication and client authentication. MongoDB is a distributed database and deployments almost always consist of multiple mongod or mongos processes running on multiple machines. Member authentication refers to the fact that these machines need to verify each other to ensure a node attempting to replicate data is indeed part of the current cluster.

On the other hand, client authentication refers to those MongoDB clients, including mongo shell, export/import tools and MongoDB drivers. Below is a deployment of standard 3 node replica set and a client.

To enable X.509 certificate based authentication, essentially you will need the following set of certificates:

• 3x server certificates, one for each MongoDB member
• 1x client certificate for one client
• 1x certificate for Root CA and 1x certificate for Signing CA

All the server certificates and client certificates must be signed by same CA.

The bulk part of the work for setting up X.509 authentication would be to create these certificates. The creation of Root CA and Signing CA and the relevant parts of the signing process is a one time effort, while the creation of the server/client certificates are as needed. For simplicity, I am going to use one machine and I will run multiple mongods on different ports for this exercise. I also put the steps into an executable script if you want to jump to the end. You can download and run the script, and you will have a test setup just like the diagram above in no time. This way you can quickly get things working, boost your confidence, and then come back to study the details.

Preparation

• One linux box, I tested on RedHat 7 and Ubuntu 14.04 but it should work with other flavors as well.
• Download MongoDB Enterprise 3.2
• Install MongoDB following instructions documented here
• Download the demo shell script and save to a clean directory

Running the script

The script will setup a test replica set on your machine and configure X.509 authentication mechanism for that replica set.

Before you run the script, double check:

• OpenSSL is installed (typically comes with most Linux distributions)
• Make sure mongod/mongo are in the executable path
• Make sure no mongod is already running on 27017 port, or change the port numbers in the shell script

To run the script, go to the directory that contains the script and execute the following:

   # chmod +x setup-x509.sh
    # ./setup-x509.sh

If everything goes smoothly, you should have a 3 nodes replica set running with X.509 as member auth. Then in the current directory, you may connect to the primary node with a newly generated client certificate client1.pem:

mongo --ssl --sslPEMKeyFile client1.pem --sslCAFile root-ca.pem --sslAllowInvalidHostnames 

Note above step just allows you to connect to the MongoDB shell. You will not have any permission at this point. To do anything meaningful, you need to authenticate yourself using following command:

$mongo
...
> db.getSiblingDB("$external").auth(
  {
    mechanism: "MONGODB-X509",
    user: "CN=client1,OU=MyClients,O=MongoDB China,L=Shenzhen,ST=GD,C=CN"
  }
);
> db.test.find()

If you are able to execute the last find statement, congratulations, the X.509 authentication is working!

Now it's time to perform an anatomy of the script to understand what are the key steps involved in setting up the X.509 authentication mechanism.

Main Parts of the Script

• Initialization
• Create local CA and signing keys
• Generate and sign server certificates for member authentication
• Generate and sign client certificates for client authentication
• Start MongoDB cluster in non-auth mode
• Setup replica set and initial users
• Restart MongoDB replica set in X.509 mode using server certificates

0. Variable initialization

First initialize some variables. Feel free to modify the values as appropriate.

dn_prefix="/C=CN/ST=GD/L=Shenzhen/O=MongoDB China"
ou_member="MyServers"
ou_client="MyClients"
mongodb_server_hosts=( "server1" "server2" "server3" )
mongodb_client_hosts=( "client1" "client2" )
mongodb_port=27017

Here dn_prefix will be used to construct the full DN name for each of the certificate. ou_member is used to have a different OU than the client certificates. Client certificates uses ou_client in its OU name.

mongodb_server_hosts should list the hostname (FQDN) for all the MongoDB servers while mongodb_client_hosts should list the hostnames for all of the client machines.

For a clean start, let’s kill the running mongods and clean up the working directory (note: don’t use -9 to kill mongod per the manual):

kill $(ps -ef | grep mongod | grep set509 | awk '{print $2}')
mkdir -p db

---

1. Create local root CA

A root CA (Certificate Authority) is at the top of the certificate chain. This is the ultimate source of the trust. Ideally a third party CA should be used. However in the case of an isolated network (very typical in large enterprise environment), or for testing purpose, we need to use local CA to test the functionality.

echo "##### STEP 1: Generate root CA "
openssl genrsa -out root-ca.key 2048
# !!! In production you will want to password protect the keys
# openssl genrsa -aes256 -out root-ca.key 2048
<p>openssl req -new -x509 -days 3650 -key root-ca.key -out root-ca.crt -subj "$dn_prefix/CN=ROOTCA"</p>
<p>mkdir -p RootCA/ca.db.certs
echo "01" >> RootCA/ca.db.serial
touch RootCA/ca.db.index
echo $RANDOM >> RootCA/ca.db.rand
mv root-ca* RootCA/

Above we first created a key pair root-ca.key with AES256 encryption and 2048 bits strength. Then using openssl req command to generate a self-signed certificate with a validity of 3650 days. One thing to call out here is the argument -x509 which tells openssl to self sign the certificate instead of generating a signing request (as what we will do below). The output is a crt file, a certificate file that contains the public key of the root CA.

---

2. Create CA config

A CA config file is used to provide some default settings during the certificate signing process, such as the directories to store the certificates etc. You may change the defaults in root-ca.cfg file after it is generated or simply change them within the script.

echo "##### STEP 2: Create CA config"
cat >> root-ca.cfg <<EOF
[ RootCA ]
dir             = ./RootCA
certs           = \$dir/ca.db.certs
database        = \$dir/ca.db.index
new_certs_dir   = \$dir/ca.db.certs
certificate     = \$dir/root-ca.crt
serial          = \$dir/ca.db.serial
private_key     = \$dir/root-ca.key
RANDFILE        = \$dir/ca.db.rand
default_md      = sha256
default_days    = 365
default_crl_days= 30
email_in_dn     = no
unique_subject  = no
policy          = policy_match
<p>[ SigningCA ]
dir             = ./SigningCA
certs           = $dir/ca.db.certs
database        = $dir/ca.db.index
new_certs_dir   = $dir/ca.db.certs
certificate     = $dir/signing-ca.crt
serial          = $dir/ca.db.serial
private_key     = $dir/signing-ca.key
RANDFILE        = $dir/ca.db.rand
default_md      = sha256
default_days    = 365
default_crl_days= 30
email_in_dn     = no
unique_subject  = no
policy          = policy_match</p>
<p>[ policy_match ]
countryName     = match
stateOrProvinceName = match
localityName            = match
organizationName    = match
organizationalUnitName  = optional
commonName      = supplied
emailAddress        = optional</p>
<p>[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment</p>
<p>[ v3_ca ]
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid:always,issuer:always
basicConstraints = CA:true
EOF

---

3. Generate the signing key

Root CA created above is typically not used for actual signing. For signing we need to delegate to so-called Subordinate Authority or Signing CA. In essence, a signing CA is just another certificate that is signed by the root CA.

echo "##### STEP 3: Generate signing key"
openssl genrsa -out signing-ca.key 2048
# again you would want to password protect your signing key:
# openssl genrsa -aes256 -out signing-ca.key 2048
<p>openssl req -new -days 1460 -key signing-ca.key -out signing-ca.csr -subj "$dn_prefix/CN=CA-SIGNER"
openssl ca -batch -name RootCA -config root-ca.cfg -extensions v3_ca -out signing-ca.crt -infiles signing-ca.csr</p>
<p>mkdir -p SigningCA/ca.db.certs
echo "01" >> SigningCA/ca.db.serial
touch SigningCA/ca.db.index</p>
<h1>Should use a better source of random here..</h1>
<p>echo $RANDOM >> SigningCA/ca.db.rand
mv signing-ca* SigningCA/

We then concatenate all the signing certificates to form a single pem file, this file will be supplied to our mongod or client process later as the value of sslCAFile parameter.

# Create root-ca.pem
cat RootCA/root-ca.crt SigningCA/signing-ca.crt > root-ca.pem

With the root CA and signing CA setup, now we're ready to sign the certificates used in MongoDB setup.

---

4. Generate and Sign server certificates

As we mentioned in the beginning, we need to separate the server certs from client certs, for the purpose of permission control.

Server certificates are intended for mongod and mongos processes. They're used for inter-member authentication.

echo "##### STEP 4: Create server certificates" 
# Pay attention to the OU part of the subject in "openssl req" command
for host in "${mongodb_server_hosts[@]}"; do
    echo "Generating key for $host"
    openssl genrsa  -out ${host}.key 2048
    openssl req -new -days 365 -key ${host}.key -out ${host}.csr -subj "$dn_prefix/OU=$ou_member/CN=${host}"
    openssl ca -batch -name SigningCA -config root-ca.cfg -out ${host}.crt -infiles ${host}.csr
    cat ${host}.crt ${host}.key > ${host}.pem   
done 

This script is in a for loop to generate multiple certificates. 3 key steps are involved with each certificate:

• Use openssl genrsa command to create a new key pair
• Use openssl req command to generate a signing request for the key
• Use openssl ca command to sign the key and output a certificate, using the Signing CA we created earlier as the signer

Notice the variable $ou_member. This signifies the difference between server certificates and client certificates. Server and client certificates must differ in the organization part of the Distinguished Names, or in another word and must differ at least in one of the O, OU, or DC values.

---

5. Generate and Sign client certificates

These certificates are used by clients, such as mongo shell, mongodump, and Java/python/C# drivers to connect to a MongoDB cluster.

This step is essentially the same as step 4 except for the use of $ou_client. This will make the combination of the DC/OU/O for these certificates will be different from the server certs above.

echo "##### STEP 5: Create client certificates"
# Pay attention to the OU part of the subject in "openssl req" command
for host in "${mongodb_client_hosts[@]}"; do
    echo "Generating key for $host"
    openssl genrsa  -out ${host}.key 2048
    openssl req -new -days 365 -key ${host}.key -out ${host}.csr -subj "$dn_prefix/OU=$ou_client/CN=${host}"
    openssl ca -batch -name SigningCA -config root-ca.cfg -out ${host}.crt -infiles ${host}.csr
    cat ${host}.crt ${host}.key > ${host}.pem
done 

---

6. Bring up replicaset in non-auth mode

MongoDB does not create a default root/admin user when enabling authentication, and there is no exception with X.509 mode. Instead, the best practice is to create an initial admin user first, then to enable authentication after the admin user has been created.

Here we're starting a replica set in non-auth mode.

echo "##### STEP 6: Start up replicaset in non-auth mode"
mport=$mongodb_port
for host in "${mongodb_server_hosts[@]}"; do
    echo "Starting server $host in non-auth mode"   
    mkdir -p ./db/${host}
    mongod --replSet set509 --port $mport --dbpath ./db/$host \
        --fork --logpath ./db/${host}.log       
    let "mport++"
done 
sleep 3

Now our replica set is up, we need to initialize the replica set and add a user.

---

7. Initialize replicaset and add initial user

When using X.509 client authentication, each client must have a user created in MongoDB and the user must be granted the necessary permissions. The username must be same as the client's DN (Distinguished Name), which can be obtained by running an openssl command:

# obtain the subject from the client key:
client_subject=`openssl x509 -in client1.pem -inform PEM -subject -nameopt RFC2253 | grep subject | awk '{sub("subject= ",""); print}'`

This would return something like:

CN=client1,OU=MyClients,O=MongoDB China,L=Shenzhen,ST=GD,C=CN

Obviously it's not mandatory to use the openssl command. It's fairly straightforward to deduce the subject string by concatenating the relevant parts of the DN as shown above.

Once we have the DN name, let's initialize the replica set and add a user:

myhostname=`hostname`
cat > setup_auth.js <<EOF
rs.initiate();
mport=$mongodb_port;
mport++;
rs.add("$myhostname:" + mport);
mport++;
rs.add("$myhostname:" + mport);
sleep(5000);
db.getSiblingDB("\$external").runCommand(
    {
        createUser: "$client_subject",
        roles: [
             { role: "readWrite", db: 'test' },
             { role: "userAdminAnyDatabase", db: "admin" },
             { role: "clusterAdmin", db:"admin"}
           ],
        writeConcern: { w: "majority" , wtimeout: 5000 }
    }
);
EOF
mongo localhost:$mongodb_port setup_auth.js 

Then stop the non-auth replica set (so that we can restart later):

kill $(ps -ef | grep mongod | grep set509 | awk '{print $2}')
sleep 3

---

8. Restart replicaset in x.509 mode

In real production, you will need to copy each of the certificate/key files to their corresponding hosts before we can start the cluster in X.509 mode. In this tutorial we are doing everything on localhost to keep thing simple.

Start all 3 nodes, note we added the following arguments:

• sslMode
• clusterAuthMode
• sslCAFile: Root CA file we created in step 2, root-ca.key
• sslPEMKeyFile: The certificate file for this host/process
• sslAllowInvalidHostnames: Only used for testing, allows invalid hostnames

Here the sslCAFile is used to establish a trust chain. As you recall the root-ca.key file contains the certificate of the root CA as well as the signing CA. By providing this file to MongoDB process, it basically tells MongoDB process to trust the certificate contained in this file, as well as all other certificates signed by these certificates contained within.

You can refer to MongoDB documentation for the details of other arguments.

echo "##### STEP 8: Restart replicaset in x.509 mode"
mport=$mongodb_port
for host in "${mongodb_server_hosts[@]}"; do
    echo "Starting server $host"    
    mongod --replSet set509 --port $mport --dbpath ./db/$host \
        --sslMode requireSSL --clusterAuthMode x509 --sslCAFile root-ca.pem \
        --sslAllowInvalidHostnames --fork --logpath ./db/${host}.log \
        --sslPEMKeyFile ${host}.pem --sslClusterFile ${host}.pem
    let "mport++"
done 

Summary

Here’s what would look like after you run the script in a clean directory:

We walked through the whole process of configuring X.509 authentication for a MongoDB replica set. The same procedure works for sharded cluster as well. A few things to keep in mind:

• The directories, Root CA and Signing CA, as well as the host itself where you generate and sign certificates for the member machines or clients, should be protected from unauthorized access.
• For simplicity, the Root CA and Signing CA keys are not password protected in this tutorial. In production it’s highly recommended to use password to protect the key from unauthorized use.
• The use of sslClusterFile parameter is for member authentication. This is optional. When absent, MongoDB will fall back to sslPEMKeyFile which can be used both for member and client authentication.

To recap the steps of creating a new certificate, either for a MongoDB cluster member host or a client host:

# substitute variables $host & $subject with actual value before executing
openssl genrsa  -out ${host}.key 2048
openssl req -new -days 365 -key ${host}.key -out ${host}.csr -subj ${subject}
openssl ca -batch -name SigningCA -config root-ca.cfg -out ${host}.crt -infiles ${host}.csr
cat ${host}.crt ${host}.key > ${host}.pem

Additional references

• Use x.509 Certificates to Authenticate Clients
• MongoDB, TLS, and x.509 Authentication Deep Dive
• Survival guides - TLS/SSL and SSL (X.509) Certificates

---

Learn more about keeping MongoDB secure.

Read the MongoDB security architecture guide

---

About the Author - TJ Tang

TJ is a Senior Solutions Architect at MongoDB Inc. TJ is based in Hong Kong and he is responsible for working with customers from a variety of industries to help them designing and architecting MongoDB enabled solutions. TJ is an experienced developer/architect with more than 15 years experience building enterprise grade software. Prior to MongoDB, he worked at FedEx Singapore as Chief Architect. TJ is an active member of open source community, he is the author of open source project Angoose, a lightweight MEAN stack implementation. When TJ is not fiddling with MongoDB, you can find him kitesurfing in the South China Sea.