Docs Menu
Docs Home
/
Enterprise Kubernetes Operator
/
Deploy Database Resources
/
Deploy

Deploy a Sharded Cluster

On this page

  • Considerations
  • Prerequisites
  • Deploy a Sharded Cluster

Note

At any place on this page that says Ops Manager, you can substitute Cloud Manager.

Important

  • You can use the Kubernetes Operator to deploy MongoDB resources with Cloud Manager and with Ops Manager version 6.0.x or later.

  • You can use the Atlas Operator to deploy MongoDB resources to Atlas.

Warning

Kubernetes Operator doesn't support arbiter nodes.

Sharded clusters provide horizontal scaling for large data sets and enable high throughput operations by distributing the data set across a group of servers.

To learn more about sharding, see Sharding Introduction in the MongoDB manual.

Use this procedure to deploy a new sharded cluster that Ops Manager manages. Later, you can use Ops Manager to add shards and perform other maintenance operations on the cluster.

Considerations

Do Not Deploy Monitoring Agents Inside and Outside Kubernetes

Due to Kubernetes network translation, a Monitoring Agent outside Kubernetes cannot monitor MongoDB instances inside Kubernetes. For this reason, k8s and non-k8s deployments in the same project are not supported. Use separate projects.

Choose Whether to Encrypt Connections

When you deploy your sharded cluster via the Kubernetes Operator, you must choose whether to encrypt connections using TLS certificates.

The following procedure for TLS-Encrypted connections:

  • Establishes TLS-encrypted connections between cluster shards.

  • Establishes TLS-encrypted connections between client applications and MongoDB deployments.

  • Requires valid certificates for TLS encryption.

The following procedure for Non-Encrypted Connections:

  • Doesn't encrypt connections between cluster shards.

  • Doesn't encrypt connections between client applications and MongoDB deployments.

  • Has fewer setup requirements than a deployment with TLS-encrypted connections.

Note

You can't secure a Standalone Instance of MongoDB in a Kubernetes cluster.

To set up TLS encryption for a replica set, see Deploy a Replica Set.

Select the appropriate tab based on whether you want to encrypt your replica set connections with TLS.

Prerequisites

To deploy a sharded cluster using an object, you must:

Note

To avoid storing secrets in single-cluster Kubernetes deployments, you can migrate all secrets to a secret storage tool. Deployments on multiple Kubernetes clusters don't support storing secrets in secret storage tools, such as HashiCorp Vault.

  • Generate one TLS certificate for each of the following components:

    • Each shard in your sharded cluster. Ensure that you add SANs for each Kubernetes pod that hosts a shard member to the certificate.

      In your TLS certificates, the SAN for each shard pod must use the following format:

      <pod-name>.<metadata.name>-sh.<namespace>.svc.cluster.local

    • Your config servers. Ensure that you add SANs for each Kubernetes pod that hosts your config servers to the certificate.

      In your TLS certificates, the SAN for each config server pod must use the following format:

      <pod-name>.<metadata.name>-cs.<namespace>.svc.cluster.local

    • Your mongos instances. Ensure that you add SANs for each Kubernetes pod that hosts a mongos to the certificate.

      In your TLS certificates, the SAN for each mongos pod must use this format:

      <pod-name>.<metadata.name>-svc.<namespace>.svc.cluster.local

    • Your project's MongoDB Agent. For the MongoDB Agent certificate, ensure that you meet the following requirements:

      • The Common Name in the TLS certificate is not empty.

      • The combined Organization and Organizational Unit in each TLS certificate differs from the Organization and Organizational Unit in the TLS certificate for your replica set members.

  • You must possess the CA certificate and the key that you used to sign your TLS certificates.

Important

The Kubernetes Operator uses kubernetes.io/tls secrets to store TLS certificates and private keys for Ops Manager and MongoDB resources. Starting in Kubernetes Operator version 1.17.0, the Kubernetes Operator doesn't support concatenated PEM files stored as Opaque secrets.

To deploy a sharded cluster using an object, you must:

Note

To avoid storing secrets in single-cluster Kubernetes deployments, you can migrate all secrets to a secret storage tool. Deployments on multiple Kubernetes clusters don't support storing secrets in secret storage tools, such as HashiCorp Vault.

Deploy a Sharded Cluster

1

Configure kubectl to default to your namespace.

If you have not already, run the following command to execute all kubectl commands in the namespace you created.

Note

If you are deploying an Ops Manager resource in a multi-Kubernetes cluster MongoDB deployment:

  • Set the context to the name of the central cluster, such as: kubectl config set context "$MDB_CENTRAL_CLUSTER_FULL_NAME".

  • Set the --namespace to the same scope that you used for your multi-Kubernetes cluster MongoDB deployment, such as: kubectl config --namespace "mongodb".

kubectl config set-context $(kubectl config current-context) --namespace=<metadata.namespace>
2

Create the secret for your Shards' TLS certificates.

Run this kubectl command to create a new secret that stores the sharded cluster shards' certificates:

kubectl -n mongodb create secret tls <prefix>-<metadata.name>-0-cert \
  --cert=<shard-0-tls-cert> \
  --key=<shard-0-tls-key>
kubectl -n mongodb create secret tls <prefix>-<metadata.name>-1-cert \
  --cert=<shard-1-tls-cert> \
  --key=<shard-1-tls-key>

If you're using HashiCorp Vault as your secret storage tool, you can Create a Vault Secret instead.

3

Create the secret for your config servers' TLS certificate.

Run this kubectl command to create a new secret that stores the sharded cluster config servers' certificate:

kubectl -n mongodb create secret tls <prefix>-<metadata.name>-config-cert \
  --cert=<config-tls-cert> \
  --key=<config-tls-key>

If you're using HashiCorp Vault as your secret storage tool, you can Create a Vault Secret instead.

4

Create the secret for your mongos servers' TLS certificate.

Run this kubectl command to create a new secret that stores the sharded cluster mongos certificate:

kubectl -n mongodb create secret tls <prefix>-<metadata.name>-mongos-cert \
  --cert=<mongos-tls-cert> \
  --key=<mongos-tls-key>

If you're using HashiCorp Vault as your secret storage tool, you can Create a Vault Secret instead.

5

Create the secret for your agent's TLS certificate.

Run this kubectl command to create a new secret that stores the agent's TLS certificate:

kubectl create secret tls <prefix>-<metadata.name>-agent-certs \
  --cert=<agent-tls-cert> \
  --key=<agent-tls-key>

If you're using HashiCorp Vault as your secret storage tool, you can Create a Vault Secret instead.

6

Create the ConfigMap to link your CA with your deployment.

Run this kubectl command to link your CA to your sharded cluster and specify the CA certificate file that you must always name ca-pem for the MongoDB resource:

kubectl create configmap custom-ca --from-file=ca-pem=<your-custom-ca-file>
7

Copy the sample sharded cluster resource.

Change the settings of this YAML file to match your desired sharded cluster configuration.

Change the settings to match your desired sharded cluster configuration.

1---
2apiVersion: mongodb.com/v1
3kind: MongoDB
4metadata:
5  name: <my-sharded-cluster>
6spec:
7  shardCount: 2
8  mongodsPerShardCount: 3
9  mongosCount: 2
10  configServerCount: 3
11  version: "8.0.0"
12  opsManager:
13    configMapRef:
14      name: <configMap.metadata.name>
15            # Must match metadata.name in ConfigMap file
16  credentials: <mycredentials>
17  type: ShardedCluster
18  persistent: true
19...
19  security:
20    tls:
21      ca: <custom-ca>
22    certsSecretPrefix: <prefix>
23...
8

Paste the copied example to create a new sharded cluster resource.

Open your preferred text editor and paste the object specification into a new text file.

9

Configure the settings highlighted in the preceding step as follows.

Key
Type
Description
Example

metadata.name

string

Label for this Kubernetes sharded cluster object.

Resource names must be 44 characters or less.

To learn more, see metadata.name and Kubernetes documentation on names.

myproject

spec.shardCount

integer

Number of shards to deploy.

2

spec.mongodsPerShardCount

integer

Number of shard members per shard.

3

spec.mongosCount

integer

Number of shard routers to deploy.

2

spec.configServerCount

integer

Number of members of the config server replica set.

3

spec.version

string

Version of MongoDB that this sharded cluster should run.

The format should be X.Y.Z for the Community edition and X.Y.Z-ent for the Enterprise edition.

IMPORTANT: Ensure that you choose a compatible MongoDB Server version. Compatible versions differ depending on the base image that the MongoDB database resource uses.

To learn more about MongoDB versioning, see MongoDB Versioning in the MongoDB Manual.

For best results, use the latest available enterprise MongoDB version that is compatible with your Ops Manager version.

spec.opsManager.configMapRef.name

string

Name of the ConfigMap with the Ops Manager connection configuration. The spec.cloudManager.configMapRef.name setting is an alias for this setting and can be used in its place.

This value must exist on the same namespace as the resource you want to create.

IMPORTANT: The Kubernetes Operator tracks any changes to the ConfigMap and reconciles the state of the MongoDB resource.

<myproject>

spec.credentials

string

Name of the secret you created as Ops Manager API authentication credentials for the Kubernetes Operator to communicate with Ops Manager.

The Ops Manager Kubernetes Secret object holding the Credentials must exist on the same Namespace as the resource you want to create.

IMPORTANT: The Kubernetes Operator tracks any changes to the Secret and reconciles the state of the MongoDB resource.

<mycredentials>

spec.type

string

Type of MongoDB resource to create.

ShardedCluster

spec.persistent

string

Optional.

Flag indicating if this MongoDB resource should use Persistent Volumes for storage. Persistent volumes are not deleted when the MongoDB resource is stopped or restarted.

If this value is true, then the following values are set to their default value of 16Gi:

To change your Persistent Volume Claims configuration, configure the following collections to meet your deployment requirements:

WARNING: Grant your containers permission to write to your Persistent Volume. The Kubernetes Operator sets fsGroup = 2000, runAsUser = 2000, and runAsNonRoot = true in securityContext. Kubernetes Operator sets fsgroup equal to runAsUser to make the volume writable for a user that runs the main process in the container. To learn more, see Configure a Security Context for a Pod or Container and the related discussion in the Kubernetes documentation. If redeploying the resource doesn't fix issues with your Persistent Volume, contact MongoDB Support.

If you do not use Persistent Volumes, the Disk Usage and Disk IOPS charts cannot be displayed in either the Processes tab on the Deployment page or in the Metrics page when reviewing the data for this deployment.

true

10

Configure the TLS settings for your sharded cluster resource using a custom certificate authority (CA).

To enable TLS in your deployment, configure the following settings in your Kubernetes object:

Key
Type
Necessity
Description
Example
spec.security
.tls.ca

string

Required

Add the ConfigMap's name that stores the custom CA that you used to sign your deployment's TLS certificates.

<custom-ca>

spec.security
.certsSecretPrefix

string

Required

Add the <prefix> of the secret name that contains your MongoDB deployment's TLS certificates.

For example, if you call your deployment my-deployment and you set the prefix to mdb, you must name the TLS secret for the client TLS communications mdb-my-deployment-cert. Also, you must name the TLS secret for internal cluster authentication (if enabled) mdb-my-deployment-clusterfile.

devDb

11

Add any additional accepted settings for a sharded cluster deployment.

You can also add any of the following optional settings to the object specification file for a sharded cluster deployment:

Warning

You must set spec.clusterDomain if your Kubernetes cluster has a default domain other than the default cluster.local. If you neither use the default nor set the spec.clusterDomain option, the Kubernetes Operator might not function as expected.

For config server

For shard routers

For shard members

12

Save this file with a .yaml file extension.

13

Start your sharded cluster deployment.

Invoke the following Kubernetes command to create your sharded cluster:

kubectl apply -f <sharded-cluster-conf>.yaml

Check the log after running this command. If the creation was successful, you should see a message similar to the following:

2018-06-26T10:30:30.346Z INFO operator/shardedclusterkube.go:52 Created! {"sharded cluster": "my-sharded-cluster"}
14

Track the status of your sharded cluster deployment.

To check the status of your MongoDB resource, use the following command:

kubectl get mdb <resource-name> -o yaml -w

With the -w (watch) flag set, when the configuration changes, the output refreshes immediately until the status phase achieves the Running state. To learn more about resource deployment statuses, see Troubleshoot the Kubernetes Operator.

After you encrypt your database resource with TLS, you can secure the following:

Renew TLS Certificates for a Sharded Cluster

Renew your TLS certificates periodically using the following procedure:

1

Configure kubectl to default to your namespace.

If you have not already, run the following command to execute all kubectl commands in the namespace you created.

Note

If you are deploying an Ops Manager resource in a multi-Kubernetes cluster MongoDB deployment:

  • Set the context to the name of the central cluster, such as: kubectl config set context "$MDB_CENTRAL_CLUSTER_FULL_NAME".

  • Set the --namespace to the same scope that you used for your multi-Kubernetes cluster MongoDB deployment, such as: kubectl config --namespace "mongodb".

kubectl config set-context $(kubectl config current-context) --namespace=<metadata.namespace>
2

Renew the secret for your Shards' TLS certificates.

Run this kubectl command to renew an existing secret that stores the sharded cluster shards' certificates:

kubectl -n mongodb create secret tls <prefix>-<metadata.name>-0-cert \
  --cert=<shard-0-tls-cert> \
  --key=<shard-0-tls-key> \
  --dry-run=client \
  -o yaml |
kubectl apply -f -
kubectl -n mongodb create secret tls <prefix>-<metadata.name>-1-cert \
  --cert=<shard-1-tls-cert> \
  --key=<shard-1-tls-key> \
  --dry-run=client \
  -o yaml |
kubectl apply -f -
3

Renew the secret for your config server's TLS certificates.

Run this kubectl command to renew an existing secret that stores the sharded cluster config server's certificates:

kubectl -n mongodb create secret tls <prefix>-<metadata.name>-config-cert \
  --cert=<config-tls-cert> \
  --key=<config-tls-key> \
  --dry-run=client \
  -o yaml |
kubectl apply -f -
4

Renew the secret for your mongos server's TLS certificates.

Run this kubectl command to renew an existing secret that stores the sharded cluster mongos certificates:

kubectl -n mongodb create secret tls <prefix>-<metadata.name>-mongos-cert \
  --cert=<mongos-tls-cert> \
  --key=<mongos-tls-key> \
  --dry-run=client \
  -o yaml |
kubectl apply -f -
1

Configure kubectl to default to your namespace.

If you have not already, run the following command to execute all kubectl commands in the namespace you created.

Note

If you are deploying an Ops Manager resource in a multi-Kubernetes cluster MongoDB deployment:

  • Set the context to the name of the central cluster, such as: kubectl config set context "$MDB_CENTRAL_CLUSTER_FULL_NAME".

  • Set the --namespace to the same scope that you used for your multi-Kubernetes cluster MongoDB deployment, such as: kubectl config --namespace "mongodb".

kubectl config set-context $(kubectl config current-context) --namespace=<metadata.namespace>
2

Copy the sample sharded cluster resource.

Change the settings of this YAML file to match your desired sharded cluster configuration.

This is a YAML file that you can modify to meet your desired configuration. Change the settings to match your desired sharded cluster configuration.

1---
2apiVersion: mongodb.com/v1
3kind: MongoDB
4metadata:
5  name: <my-sharded-cluster>
6spec:
7  shardCount: 2
8  mongodsPerShardCount: 3
9  mongosCount: 2
10  configServerCount: 3
11  version: "8.0.0"
12  opsManager:
13    configMapRef:
14      name: <configMap.metadata.name>
15            # Must match metadata.name in ConfigMap file
16  credentials: <mycredentials>
17  type: ShardedCluster
18  persistent: true
19...
3

Paste the copied example to create a new sharded cluster resource.

Open your preferred text editor and paste the object specification into a new text file.

4

Configure the settings highlighted in the preceding step as follows.

Key
Type
Description
Example

metadata.name

string

Label for this Kubernetes sharded cluster object.

Resource names must be 44 characters or less.

To learn more, see metadata.name and Kubernetes documentation on names.

myproject

spec.shardCount

integer

Number of shards to deploy.

2

spec.mongodsPerShardCount

integer

Number of shard members per shard.

3

spec.mongosCount

integer

Number of shard routers to deploy.

2

spec.configServerCount

integer

Number of members of the config server replica set.

3

spec.version

string

Version of MongoDB that this sharded cluster should run.

The format should be X.Y.Z for the Community edition and X.Y.Z-ent for the Enterprise edition.

IMPORTANT: Ensure that you choose a compatible MongoDB Server version. Compatible versions differ depending on the base image that the MongoDB database resource uses.

To learn more about MongoDB versioning, see MongoDB Versioning in the MongoDB Manual.

For best results, use the latest available enterprise MongoDB version that is compatible with your Ops Manager version.

spec.opsManager.configMapRef.name

string

Name of the ConfigMap with the Ops Manager connection configuration. The spec.cloudManager.configMapRef.name setting is an alias for this setting and can be used in its place.

This value must exist on the same namespace as the resource you want to create.

IMPORTANT: The Kubernetes Operator tracks any changes to the ConfigMap and reconciles the state of the MongoDB resource.

<myproject>

spec.credentials

string

Name of the secret you created as Ops Manager API authentication credentials for the Kubernetes Operator to communicate with Ops Manager.

The Ops Manager Kubernetes Secret object holding the Credentials must exist on the same Namespace as the resource you want to create.

IMPORTANT: The Kubernetes Operator tracks any changes to the Secret and reconciles the state of the MongoDB resource.

<mycredentials>

spec.type

string

Type of MongoDB resource to create.

ShardedCluster

spec.persistent

string

Optional.

Flag indicating if this MongoDB resource should use Persistent Volumes for storage. Persistent volumes are not deleted when the MongoDB resource is stopped or restarted.

If this value is true, then the following values are set to their default value of 16Gi:

To change your Persistent Volume Claims configuration, configure the following collections to meet your deployment requirements:

WARNING: Grant your containers permission to write to your Persistent Volume. The Kubernetes Operator sets fsGroup = 2000, runAsUser = 2000, and runAsNonRoot = true in securityContext. Kubernetes Operator sets fsgroup equal to runAsUser to make the volume writable for a user that runs the main process in the container. To learn more, see Configure a Security Context for a Pod or Container and the related discussion in the Kubernetes documentation. If redeploying the resource doesn't fix issues with your Persistent Volume, contact MongoDB Support.

If you do not use Persistent Volumes, the Disk Usage and Disk IOPS charts cannot be displayed in either the Processes tab on the Deployment page or in the Metrics page when reviewing the data for this deployment.

true

5

Add any additional accepted settings for a sharded cluster deployment.

You can also add any of the following optional settings to the object specification file for a sharded cluster deployment:

Warning

You must set spec.clusterDomain if your Kubernetes cluster has a default domain other than the default cluster.local. If you neither use the default nor set the spec.clusterDomain option, the Kubernetes Operator might not function as expected.

For config server

For shard routers

For shard members

6

Save this file with a .yaml file extension.

7

Start your sharded cluster deployment.

Invoke the following Kubernetes command to create your sharded cluster:

kubectl apply -f <sharded-cluster-conf>.yaml

Check the log after running this command. If the creation was successful, you should see a message similar to the following:

2018-06-26T10:30:30.346Z INFO operator/shardedclusterkube.go:52 Created! {"sharded cluster": "my-sharded-cluster"}
8

Track the status of your sharded cluster deployment.

To check the status of your MongoDB resource, use the following command:

kubectl get mdb <resource-name> -o yaml -w

With the -w (watch) flag set, when the configuration changes, the output refreshes immediately until the status phase achieves the Running state. To learn more about resource deployment statuses, see Troubleshoot the Kubernetes Operator.

Back

Replica Set

Next

Use with Prometheus