Run Secure Containerized MongoDB Deployments Using the MongoDB Community Kubernetes Operator
First introduced earlier this year, the MongoDB Community Kubernetes Operator now allows you to run secure MongoDB deployments in your Kubernetes cluster.
The Community Operator is open source, and ideally suited for experimentation, testing, and lightweight production use cases. For larger or more mission-critical workloads with requirements around monitoring, alerting, and data recovery, we recommend the MongoDB Enterprise Kubernetes Operator, available with MongoDB Enterprise Advanced.
This blog tutorial will show you how to deploy and configure a fully secure MongoDB deployment inside Kubernetes from scratch, using the MongoDB Community Kubernetes Operator and cert-manager.
The Community Operator is available
here
.
Installation
The MongoDB Community Kubernetes Operator allows you to deploy secure MongoDB Replica Sets in your Kubernetes cluster.
Before we can deploy MongoDB, we need to ensure that we have created the required CustomResourceDefinition.
Note: This operation requires cluster admin permissions.
kubectl apply -f https://raw.githubusercontent.com/mongodb/mongodb-kubernetes-operator/master/config/crd/bases/mongodbcommunity.mongodb.com_mongodbcommunity.yaml
Create a namespace for our deployment.
kubectl create namespace mongodb
Install the latest version of the operator.
kubectl apply -f https://raw.githubusercontent.com/mongodb/mongodb-kubernetes-operator/master/config/crd/bases/mongodbcommunity.mongodb.com_mongodbcommunity.yaml
Add authorizations.
kubectl apply -f https://raw.githubusercontent.com/mongodb/mongodb-kubernetes-operator/master/config/rbac/role_binding.yaml
kubectl apply -f https://raw.githubusercontent.com/mongodb/mongodb-kubernetes-operator/master/config/rbac/service_account.yaml
kubectl apply -f https://raw.githubusercontent.com/mongodb/mongodb-kubernetes-operator/master/config/rbac/role.yaml
*Note: If using OpenShift, make sure to reference the
OpenShift samples
instead.
Deploying a SCRAM Enabled Replica Set
The Community Operator creates secure SCRAM-SHA-256 enabled deployments by default. This means that we need to define our user and what roles we want them to have, alongside a set of credentials for the user to use.
We can create the user's credentials in the form of a Kubernetes Secret
kubectl create secret generic my-mongodb-user-password -n mongodb --from-literal="password=TXs3ZsuIqT-pQFvwxOec"
Once we have created the secret, we can deploy a MongoDB replica set.
---
apiVersion: mongodbcommunity.mongodb.com/v1
kind: MongoDBCommunity
metadata:
name: mongodb-replica-set
namespace: mongodb
spec:
members: 3
type: ReplicaSet
version: "4.4.0"
security:
authentication:
modes: ["SCRAM"]
users:
- name: my-mongodb-user
db: admin
passwordSecretRef:
name: my-mongodb-user-password # the name of the secret we created
roles: # the roles that we want to the user to have
- name: readWrite
db: myDb
scramCredentialsSecretName: mongodb-replica-set
Note:
If your application is in the same namespace, it can use this secret to connect to the MongoDB instance. If your application gets the credentials some other way, this secret can be deleted.
If you want to change this user's password in the future, you can simply create a new secret with the same name, or reference a different secret in the resource definition.
Once the MongoDBCommunity resource has been created, the operator will create and configure a StatefulSet for this replica set.
You'll notice that each pod consists of 2 containers, the mongod itself, and the mongodb-agent which runs in a sidecar and handles automation of the mongod processes.
Once the MongoDBCommunity resource has been created, we can wait for the replica set to get into the “Running” state.
NAME READY STATUS RESTARTS AGE
mongodb-kubernetes-operator-5d757df5c8-d6ll7 1/1 Running 0 5m21s
mongodb-replica-set-0 2/2 Running 0 2m56s
mongodb-replica-set-1 2/2 Running 0 2m10s
mongodb-replica-set-2 2/2 Running 0 72s
Connecting to the Replica Set
Once the resource has been successfully created, we can connect and authenticate to the MongoDB replica set as the user we defined in the resource specification.
Now you can connect to the replica set from your application using the following connection string:
USERNAME_DB="my-mongodb-user"
PASSWORD="$(kubectl get secret my-mongodb-user-password -o jsonpath='{.data.password}' | base64 -d)"
CONNECTION_STRING="mongodb://${USERNAME_DB}:${PASSWORD}@mongodb-replica-set-0.mongodb-replica-set-svc.mongodb.svc.cluster.local:27017,mongodb-replica-set-1.mongodb-replica-set-svc.mongodb.svc.cluster.local:27017,mongodb-replica-set-2.mongodb-replica-set-svc.mongodb.svc.cluster.local:27017"
We can also connect directly through the mongo shell.
MONGO_URI="$(kubectl get mdbc mongodb-replica-set -o jsonpath='{.status.mongoUri}')"
kubectl exec -it mongodb-replica-set-0 -c mongod -- mongo ${MONGO_URI} --username_db "${USERNAME_DB}" --password "${PASSWORD}"
Note: As our user only has access to the "myDb" database, we only have permissions to read and write to this database.
use myDb
db.col.insert({
"hello": "world"
})
Configure TLS with Jetstack's Cert Manager
Cert-manager
is a Kubernetes add-on from Jetstack which automates the management and issuing of TLS certificates.
The Community Operator is fully compatible with the cert-manager certificate format.
Install cert-manager into your cluster:
https://cert-manager.io/docs/installation/kubernetes/
Generate a certificate authority that will issue the certificates for our replica set.
openssl genrsa -out ca.key 2048
Generate a CA certificate, or use your own.
Note: If using your own CA certificate, you'll need to make sure a few requirements are met.
It has a filename of "ca.crt"
The common name either matches the domain name of all replica set members or has the domain name of all replica set members. View the
docs
for more details.
When generating the CA certificate, we'll use a wildcard Common Name that matches the domain name of all of the replica set members.
COMMON_NAME="*.mongodb-replica-set-svc.mongodb.svc.cluster.local"
openssl req -x509 -new -nodes -key ca.key -subj "/CN=${COMMON_NAME}" -days 3650 -reqexts v3_req -extensions v3_ca -out ca.crt
Create a kubernetes configmap containing the CA.
kubectl create configmap ca-config-map --from-file=ca.crt --namespace mongodb
Create a kubernetes secret containing the signing pair in the mongodb namespace.
kubectl create secret tls ca-key-pair --cert=ca.crt --key=ca.key --namespace=mongodb
Once we have created our key pair, we can create a cert-manager issuer resource from our key pair which will issue the certificates for our MongoDB deployment.
Create the following cert-manager issuer custom resource definition and save it as cert-manager-issuer.yaml.
---
apiVersion: cert-manager.io/v1alpha2
kind: Issuer
metadata:
name: ca-issuer
namespace: mongodb
spec:
ca:
secretName: ca-key-pair
Next, apply the resource definition.
kubectl apply -f cert-manager-issuer.yaml
Create a cert-manager certificate resource definition which references the newly created issuer into cert-manager-certificate.yaml.
---
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: cert-manager-certificate
namespace: mongodb
spec:
secretName: mongodb-tls
issuerRef:
name: ca-issuer
kind: Issuer
commonName: "*.mongodb-replica-set-svc.mongodb.svc.cluster.local"
organization:
- MongoDB
Apply the certificate resource.
kubectl apply -f cert-manager-certificate.yaml
Shortly after we create the Certificate resource, we should see the "mongodb-tls" secret which was created by cert-manager.
kubectl get secret mongodb-tls
NAME TYPE DATA AGE
mongodb-tls kubernetes.io/tls 3 1m
Without making any modifications to the secret, we can update our MongoDB resource to configure TLS.
We just need to reference both the configmap containing the ca, and the secret that cert-manager generated containing the certificates for our deployment.
---
apiVersion: mongodb.com/v1
kind: MongoDBCommunity
metadata:
name: mongodb-replica-set
namespace: mongodb
spec:
members: 3
type: ReplicaSet
version: 4.4.0
security:
tls:
enabled: true
certificateKeySecretRef:
name: mongodb-tls
caConfigMapRef:
name: ca-config-map
authentication:
modes:
- SCRAM
users:
- name: my-mongodb-user
db: admin
passwordSecretRef:
name: my-user-password
roles:
- name: readWrite
db: myDb
scramCredentialsSecretName: mongodb-replica-set
NAME READY STATUS RESTARTS AGE
mongodb-kubernetes-operator-5d757df5c8-d6ll7 1/1 Running 0 3h16m
mongodb-replica-set-0 2/2 Running 1 13m
mongodb-replica-set-1 2/2 Running 1 13m
mongodb-replica-set-2 2/2 Running 1 12m
Shortly after the updated MongoDBCommunity resource was applied, we should see the members of the replica set back in the “Running” state. Each member will have been restarted once as changing TLS configuration results in a rolling restart of the deployment.
Once the changes have all been applied, we can test our connection over TLS by connecting to any of the mongod containers:
kubectl exec -it mongodb-replica-set-0 -c mongod -- bash
And using the mongo shell to connect using TLS:
mongo --tls --tlsCAFile /var/lib/tls/ca/ca.crt --tlsCertificateKeyFile /var/lib/tls/server/*.pem --host mongodb-replica-set-0.mongodb-replica-set-svc.mongodb.svc.cluster.local
In this blog tutorial we:
Deployed the MongoDB Community Kubernetes Operator into our Kubernetes cluster
Created a secure, SCRAM-SHA enabled MongoDBCommunity resource and our password in the form of a Kubernetes secret
Used cert-manager to create TLS certificates for our MongoDB deployment
And finally, configured our MongoDB resource to enable TLS for our deployment
To get started yourself, download MongoDB Community Kubernetes Operator
here
.
December 14, 2020