Navigation
This version of the documentation is archived and no longer supported. To learn how to upgrade your version of MongoDB Kubernetes Operator, refer to the upgrade documentation.

Deploy a Sharded Cluster

On this page

Supported with Cloud Manager and Ops Manager 4.0

You can use Kubernetes Operator to deploy MongoDB instances with Ops Manager version 4.0 or later and Cloud Manager. At any place in this guide that says Ops Manager, you can substitute Cloud Manager.

Sharded clusters provide horizontal scaling for large data sets and enable high throughput operations by distributing the data set across a group of servers.

To learn more about sharding, see Sharding Introduction in the MongoDB manual.

Use this procedure to deploy a new sharded cluster that Ops Manager manages. Later, you can use Ops Manager to add shards and perform other maintenance operations on the cluster.

Prerequisites

To deploy a sharded cluster using an object, you need to complete the following procedures:

Considerations

Use unique names for the new cluster and its shards.

Important

Replica set, sharded cluster, and shard names within the same project must be unique. Failure to have unique names for the deployments will result in broken backup snapshots.

Do Not Deploy Monitoring Agents inside and outside Kubernetes

Do not mix MongoDB deployments outside Kubernetes with ones insider Kubernetes in the same Project.

Due to Kubernetes network translation, a Monitoring Agent outside Kubernetes cannot monitor MongoDB instnaces inside Kubernetes. For this reason, k8s and non-k8s deployments in the same Project is not supported. Use separate projects.

Procedure

The procedure for deploying a sharded cluster depends on whether you require the deployment to run with TLS enabled for intra-cluster communication and clients connecting to the database:

1

Copy the following example sharded cluster Kubernetes object specification.

This is a YAML file that you can modify to meet your desired configuration. Change the highlighted settings to match your desired sharded cluster configuration.

copy 
---
apiVersion: mongodb.com/v1
kind: MongoDB
metadata:
  name: <myshardedcluster>
  namespace: <metadata.namespace> # Should match metadata.namespace in
                                  # your configmap file.
spec:
  shardCount: 2
  mongodsPerShardCount: 3
  mongosCount: 2
  configServerCount: 3
  version: 4.0.6
  project: <myconfigmap> # Should match metadata.name in your
                         # configmap file.
  credentials: <mycredentials>
  type: ShardedCluster
  persistent: true
...
2

Open your preferred text editor and paste the object specification into a new text file.

3

Configure the settings highlighted in the preceding step as follows.

Key Type Description Example
metadata.name string

Label for this Kubernetes sharded cluster object.

See also

 myproject
metadata.namespace string

Scope of object names. Kubernetes namespace where this MongoDB Kubernetes resource and other objects are created.

Using two different namespaces allows you to delete your sharded cluster or all of the resources in the namespace without affecting your Kubernetes Operator.

See also

 mongodb
spec.shardCount integer Number of shards to deploy. 2
spec.mongodsPerShardCount integer Number of shard members per shard. 3
spec.mongosCount integer Number of shard routers to deploy. 2
spec.configServerCount integer Number of members of the config server replica set. 3
spec.version string

Version of MongoDB that this sharded cluster should run.

The format should be X.Y.Z for the Community edition and X.Y.Z-ent for the Enterprise edition.

To learn more about MongoDB versioning, see MongoDB Versioning in the MongoDB Manual.

 3.6.7
spec.project string

Name of the ConfigMap with the Ops Manager connection configuration.

Value must match namespace and name of ConfigMap

This value must match the value you provided for metadata.name in your Ops Manager project ConfigMap.

If this MongoDB Kubernetes resource is in a different namespace than the project ConfigMap, you should set this value to the namespace and name of the ConfigMap in this format: <namespace>/<myconfigmap>

Operator manages changes to the ConfigMap

The Kubernetes Operator tracks any changes to the ConfigMap and reconciles the state of the MongoDB Kubernetes resource.

 <myproject> or <namespace>/<myconfigmap>
spec.credentials string

Name of the Kubernetes secret you created as Ops Manager API authentication credentials for the Kubernetes Operator to communicate with Ops Manager.

Value must use namespace and name of Secret

This value must match the value you provided for namespace and name for your Ops Manager Kubernetes Secret.

If this object is in a different namespace than the Secret, you should set this value to the namespace and name of the Secret in this format: <namespace>/<name>

Operator manages changes to the Secret

The Kubernetes Operator tracks any changes to the Secret and reconciles the state of the MongoDB Kubernetes resource.

 <mycredentials> or <namespace>/<mycredentials>
spec.type string Type of MongoDB Kubernetes resource to create. ShardedCluster
spec.persistent string

Optional.

Flag indicating if this MongoDB Kubernetes resource should use Persistent Volumes for storage. Persistent volumes are not deleted when the MongoDB Kubernetes resource is stopped or restarted.

If this value is true, then the following values are set to their default value of 16G:

To change your Persistent Volume Claims configuration, configure the following collections to meet your deployment requirements:

Warning

Your containers must have permissions to write to your Persistent Volume. The Kubernetes Operator sets fsGroup = 2000 in securityContext This makes Kubernetes try to fix write permissions for the Persistent Volume. If redeploying the deployment item does not fix issues with your Persistent Volumes, contact MongoDB support.

Note

If you do not use Persistent Volumes, the Disk Usage and Disk IOPS charts cannot be displayed in either the Processes tab on the Deployment page or in the Metrics page when reviewing the data for this deployment.

 true
4
5

Save this file with a .yaml file extension.

6

Start your sharded cluster deployment.

Invoke the following Kubernetes command to create your sharded cluster:

copy 
kubectl apply -f <sharded-cluster-conf>.yaml

Check the log after running this command. If the creation was successful, you should see a message similar to the following:

copy 
2018-06-26T10:30:30.346Z INFO operator/shardedclusterkube.go:52 Created! {"sharded cluster": "my-sharded-cluster"}
7

Track the status of your sharded cluster deployment.

To check the status of your MongoDB Kubernetes resource, invoke the following command:

copy 
kubectl get mdb <resource-name> -n <namespace> -o yaml -w

The -w flag means “watch”. With the “watch” flag set, the output refreshes immediately when something changes until the status phase achieves the Running state.

If the deployment fails, see Troubleshooting the Kubernetes Operator.

The MongoDB Enterprise Kubernetes Operator can use TLS certificates to encrypt communication between:

  • MongoDB hosts in a replica set or sharded cluster configuration
  • Clients (mongo shell, drivers, MongoDB Compass, and others) and the MongoDB deployment

The following procedure walks you through deploying a replica set with TLS enabled:

1

Copy the following example sharded cluster Kubernetes object specification.

This is a YAML file that you can modify to meet your desired configuration. Change the highlighted settings to match your desired sharded cluster configuration.

copy 
---
apiVersion: mongodb.com/v1
kind: MongoDB
metadata:
  name: my-secure-sc
  namespace: <metadata.namespace> # Should match metadata.namespace in
                                  # your configmap file.
spec:
  shardCount: 2
  mongodsPerShardCount: 3
  mongosCount: 2
  configServerCount: 3
  version: 4.0.6
  project: <myconfigmap> # Should match metadata.name in your
                         # configmap file.
  credentials: <mycredentials>
  type: ShardedCluster
  persistent: true
  security:
    tls:
      enabled: true
    clusterAuthenticationMode: x509 # Optional. Enables X.509 internal
                                    # cluster authentication.
  additionalMongodConfig: # Optional
    net:
      ssl:
        mode: requireSSL
...
2

Open your preferred text editor and paste the object specification into a new text file.

3

Configure the settings highlighted in the preceeding step as follows.

Key Type Description Example
metadata.name string

Label for this Kubernetes sharded cluster object.

See also

 myproject
metadata.namespace string

Scope of object names. Kubernetes namespace where this MongoDB Kubernetes resource and other objects are created.

Using two different namespaces allows you to delete your sharded cluster or all of the resources in the namespace without affecting your Kubernetes Operator.

See also

 mongodb
spec.shardCount integer Number of shards to deploy. 2
spec.mongodsPerShardCount integer Number of shard members per shard. 3
spec.mongosCount integer Number of shard routers to deploy. 2
spec.configServerCount integer Number of members of the config server replica set. 3
spec.version string

Version of MongoDB that this sharded cluster should run.

The format should be X.Y.Z for the Community edition and X.Y.Z-ent for the Enterprise edition.

To learn more about MongoDB versioning, see MongoDB Versioning in the MongoDB Manual.

 3.6.7
spec.project string

Name of the ConfigMap with the Ops Manager connection configuration.

Value must match namespace and name of ConfigMap

This value must match the value you provided for metadata.name in your Ops Manager project ConfigMap.

If this MongoDB Kubernetes resource is in a different namespace than the project ConfigMap, you should set this value to the namespace and name of the ConfigMap in this format: <namespace>/<myconfigmap>

Operator manages changes to the ConfigMap

The Kubernetes Operator tracks any changes to the ConfigMap and reconciles the state of the MongoDB Kubernetes resource.

 <myproject> or <namespace>/<myconfigmap>
spec.credentials string

Name of the Kubernetes secret you created as Ops Manager API authentication credentials for the Kubernetes Operator to communicate with Ops Manager.

Value must use namespace and name of Secret

This value must match the value you provided for namespace and name for your Ops Manager Kubernetes Secret.

If this object is in a different namespace than the Secret, you should set this value to the namespace and name of the Secret in this format: <namespace>/<name>

Operator manages changes to the Secret

The Kubernetes Operator tracks any changes to the Secret and reconciles the state of the MongoDB Kubernetes resource.

 <mycredentials> or <namespace>/<mycredentials>
spec.type string Type of MongoDB Kubernetes resource to create. ShardedCluster
spec.persistent string

Optional.

Flag indicating if this MongoDB Kubernetes resource should use Persistent Volumes for storage. Persistent volumes are not deleted when the MongoDB Kubernetes resource is stopped or restarted.

If this value is true, then the following values are set to their default value of 16G:

To change your Persistent Volume Claims configuration, configure the following collections to meet your deployment requirements:

Warning

Your containers must have permissions to write to your Persistent Volume. The Kubernetes Operator sets fsGroup = 2000 in securityContext This makes Kubernetes try to fix write permissions for the Persistent Volume. If redeploying the deployment item does not fix issues with your Persistent Volumes, contact MongoDB support.

Note

If you do not use Persistent Volumes, the Disk Usage and Disk IOPS charts cannot be displayed in either the Processes tab on the Deployment page or in the Metrics page when reviewing the data for this deployment.

 true
4

If you wish to enable TLS on your sharded cluster, add the TLS settings to the ConfigMap.

Enable TLS in your deployment by configuring the following settings in your Kubernetes object:

Key Type Description Example
spec.security.tls.enabled boolean

If this value is true, TLS is enabled on the MongoDB deployment.

The default TLS mode is requireSSL which configures the hosts to only use and accept TLS encrypted connections.

 true
spec.security.clusterAuthenticationMode string

Optional. Enables X.509 internal cluster authentication.

Remove this field from your ConfigMap to disable X.509 internal cluster authentication.

Important

Once internal cluster authentication is enabled, it can not be disabled.

 x509
spec.additionalMongodConfig.net.ssl.mode string Optional. Changes the TLS mode the MongoDB deployment uses for intra-cluster communication and clients connecting to the database. requireSSL

The following options are valid for spec.additionalMongodConfig.net.ssl.mode:

Value Description
allowSSL Connections between servers do not use TLS. For incoming connections, the server accepts both TLS and non-TLS.
preferSSL Connections between servers use TLS. For incoming connections, the server accepts both TLS and non-TLS.
requireSSL The server uses and accepts only TLS encrypted connections.

Note

MongoDB custom resources do not support all mongod command line options. If you use an unsupported option in your object specification file, the backing MongoDB Agent overrides the unsupported options. For a complete list of options supported by MongoDB custom resources, see MongoDB Kubernetes Object Specification.

5
6

Save this file with a .yaml file extension.

7

Start your sharded cluster deployment.

Invoke the following Kubernetes command to create your sharded cluster:

copy 
kubectl apply -f <sharded-cluster-conf>.yaml
8

Check the status of your deployment.

The Kubernetes Operator will proceed to create the MongoDB resources and request the Kubernetes Certificate Authority to approve the database host’s certificates. Verify that the certificates are pending approval by running the following command:

copy 
kubectl get mdb <resource-name> -n <namespace> -o yaml -w

The status field of the output should resemble the following:

status:
  lastTransition: 2019-05-01T15:36:59Z
  message: Not all certificates have been approved by Kubernetes CA
  phase: Failed
  type: ""
  version: ""

If you do not see the status.message above, see Troubleshooting the Kubernetes Operator to help diagnose the issue.

9

Approve the certificate for each host in your deployment.

Retrieve the CSRs for each host by running the following command:

copy 
kubectl get csr

The output of the command and number of certificates to approve depend on whether X.509 internal cluster authentication is enabled by setting spec.security.clusterAuthenticationMode to x509 in step 4.

The command’s output resembles the following:

copy 
NAME                                       AGE       REQUESTOR                                                   CONDITION
my-secure-sc-0-0.mongodb                   30s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-0-1.mongodb                   28s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-0-2.mongodb                   27s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-1-0.mongodb                   22s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-1-1.mongodb                   13s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-1-2.mongodb                   6s        system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-config-0.mongodb              36s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-config-1.mongodb              34s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-config-2.mongodb              32s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-mongos-0.mongodb              49s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-mongos-1.mongodb              42s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending

Using the NAME field above, approve each certificate from the previous command’s output using the following command:

copy 
kubectl certificate approve <NAME>

Example

The following commands approve the certificates for the replica set example in the previous step:

kubectl certificate approve my-secure-sc-0-0.mongodb
kubectl certificate approve my-secure-sc-0-1.mongodb
kubectl certificate approve my-secure-sc-0-2.mongodb
kubectl certificate approve my-secure-sc-1-0.mongodb
kubectl certificate approve my-secure-sc-1-1.mongodb
kubectl certificate approve my-secure-sc-1-2.mongodb
kubectl certificate approve my-secure-sc-config-0.mongodb
kubectl certificate approve my-secure-sc-config-1.mongodb
kubectl certificate approve my-secure-sc-config-2.mongodb
kubectl certificate approve my-secure-sc-mongos-0.mongodb
kubectl certificate approve my-secure-sc-mongos-1.mongodb

kubectl prints a message to the console when a certificate is approved.

The command’s output resembles the following:

copy 
NAME                                       AGE       REQUESTOR                                                   CONDITION
mms-automation-agent.mongodb               20m       system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
mms-backup-agent.mongodb                   20m       system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
mms-monitoring-agent.mongodb               20m       system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
my-secure-sc-0-0.mongodb                   30s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-0-1.mongodb                   28s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-0-2.mongodb                   27s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-1-0.mongodb                   22s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-1-1.mongodb                   13s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-1-2.mongodb                   6s        system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-config-0.mongodb              36s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-config-1.mongodb              34s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-config-2.mongodb              32s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-mongos-0.mongodb              49s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-mongos-1.mongodb              42s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending

Using the NAME field above, approve each certificate from the previous command’s output using the following command:

copy 
kubectl certificate approve <NAME>

Example

The following commands approve the certificates for the replica set example in the previous step:

kubectl certificate approve my-secure-sc-0-0.mongodb
kubectl certificate approve my-secure-sc-0-1.mongodb
kubectl certificate approve my-secure-sc-0-2.mongodb
kubectl certificate approve my-secure-sc-1-0.mongodb
kubectl certificate approve my-secure-sc-1-1.mongodb
kubectl certificate approve my-secure-sc-1-2.mongodb
kubectl certificate approve my-secure-sc-config-0.mongodb
kubectl certificate approve my-secure-sc-config-1.mongodb
kubectl certificate approve my-secure-sc-config-2.mongodb
kubectl certificate approve my-secure-sc-mongos-0.mongodb
kubectl certificate approve my-secure-sc-mongos-1.mongodb

kubectl prints a message to the console when a certificate is approved.

When spec.security.clusterAuthenticationMode is set to x509 an additional CSR will be generated per host for the clusterfile.

After the first batch of certificates are approved, run the command to retrieve the CSRs again:

copy 
kubectl get csr

The clusterfile CSRs are now present in the output:

copy 
NAME                                        AGE       REQUESTOR                                                   CONDITION
mms-automation-agent.mongodb                22m       system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
mms-backup-agent.mongodb                    22m       system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
mms-monitoring-agent.mongodb                22m       system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
my-secure-sc-0-0-clusterfile.mongodb        40s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-0-0.mongodb                    2m22s     system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
my-secure-sc-0-1-clusterfile.mongodb        36s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-0-1.mongodb                    2m20s     system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
my-secure-sc-0-2-clusterfile.mongodb        32s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-0-2.mongodb                    2m19s     system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
my-secure-sc-1-0-clusterfile.mongodb        27s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-1-0.mongodb                    2m14s     system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
my-secure-sc-1-1-clusterfile.mongodb        23s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-1-1.mongodb                    2m5s      system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
my-secure-sc-1-2-clusterfile.mongodb        20s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-1-2.mongodb                    118s      system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
my-secure-sc-config-0-clusterfile.mongodb   10s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-config-0.mongodb               2m28s     system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
my-secure-sc-config-1-clusterfile.mongodb   5s        system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-config-1.mongodb               2m26s     system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
my-secure-sc-config-2-clusterfile.mongodb   2s        system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-config-2.mongodb               2m24s     system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
my-secure-sc-mongos-0-clusterfile.mongodb   18s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-mongos-0.mongodb               2m41s     system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved,Issued
my-secure-sc-mongos-1-clusterfile.mongodb   12s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-sc-mongos-1.mongodb               2m34s     system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending

Approve the clusterfile CSRs using the same command:

copy 
kubectl certificate approve <NAME>

Example

The following commands approve the clusterfile certificates:

kubectl certificate approve my-secure-sc-0-0-clusterfile.mongodb
kubectl certificate approve my-secure-sc-0-1-clusterfile.mongodb
kubectl certificate approve my-secure-sc-0-2-clusterfile.mongodb
kubectl certificate approve my-secure-sc-1-0-clusterfile.mongodb
kubectl certificate approve my-secure-sc-1-1-clusterfile.mongodb
kubectl certificate approve my-secure-sc-1-2-clusterfile.mongodb
kubectl certificate approve my-secure-sc-config-0-clusterfile.mongodb
kubectl certificate approve my-secure-sc-config-1-clusterfile.mongodb
kubectl certificate approve my-secure-sc-config-2-clusterfile.mongodb
kubectl certificate approve my-secure-sc-mongos-0-clusterfile.mongodb
kubectl certificate approve my-secure-sc-mongos-1-clusterfile.mongodb
10

Track the status of your sharded cluster deployment.

To check the status of your MongoDB Kubernetes resource, invoke the following command:

copy 
kubectl get mdb <resource-name> -n <namespace> -o yaml -w

The -w flag means “watch”. With the “watch” flag set, the output refreshes immediately when something changes until the status phase achieves the Running state.

If the deployment fails, see Troubleshooting the Kubernetes Operator.

© MongoDB, Inc 2008-present. MongoDB, Mongo, and the leaf logo are registered trademarks of MongoDB, Inc.