> For the complete MongoDB documentation index, see www.mongodb.com/docs/llms.txt

# Best Practices

This page details best practices and system configuration recommendations for the MongoDB Controllers for Kubernetes Operator when running in production.

## Deploy Multiple MongoDB Replica Sets

We recommend that you use a single instance of the Kubernetes Operator to deploy and manage your MongoDB replica sets.

To deploy more than 10 MongoDB replica sets in parallel, you can [increase the thread count of your Kubernetes Operator instance.](https://www.mongodb.com/docs/kubernetes/current/tutorial/plan-k8s-op-considerations.md#std-label-increase-thread-count-ops-manager)

## Set Scope for MongoDB Controllers for Kubernetes Operator Deployment

Before you install the Kubernetes Operator, you can set the scope of the Kubernetes Operator deployment. The scopes depend on the namespaces in which you choose to deploy Ops Manager and `MongoDB` resources.

### Kubernetes Operator Deployment Scopes

You can set one of these scopes:

- [Operator Uses the Same Single Namespace as Resources](https://www.mongodb.com/docs/kubernetes/current/tutorial/plan-k8s-op-considerations.md#std-label-ns-scope-same-ns) *(Default)*

- [Operator Uses a Subset of Namespaces](https://www.mongodb.com/docs/kubernetes/current/tutorial/plan-k8s-op-considerations.md#std-label-ns-scope-different-ns)

- [Operator Uses Cluster-Wide Scope](https://www.mongodb.com/docs/kubernetes/current/tutorial/plan-k8s-op-considerations.md#std-label-cluster-wide-scope)

#### Operator Uses the Same Single Namespace as Resources

You can set the scope for the Kubernetes Operator to use the same [namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) as resources. In this case, the Kubernetes Operator watches Ops Manager and `MongoDB` resources in that same [namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/).

When you [install](https://www.mongodb.com/docs/kubernetes/current/tutorial/install-k8s-operator.md#std-label-install-k8s-operator) the Kubernetes Operator, it uses the default namespace.

#### Operator Uses a Subset of Namespaces

You can set the scope for the Kubernetes Operator to use one or more [namespaces](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) that differ from the namespace used by the Kubernetes Operator resources. In this case, the Kubernetes Operator watches Ops Manager and `MongoDB` resources in a subset of [namespaces](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) that you specify.

To install the Kubernetes Operator instances with this scope, use `helm` with the [operator.watchNamespace](https://www.mongodb.com/docs/kubernetes/current/reference/helm-operator-settings.md#std-label-helm-watch-namespace) parameter.

Watching a subset of namespaces is useful in deployments where a single Kubernetes Operator instance watches a different cluster resource type. For example, you can configure the Kubernetes Operator to watch `MongoDB` resources in one subset of namespaces, and to watch `MongoDBMultiCluster` resources in another subset of namespaces. To avoid race conditions during resource reconciliation, for each custom resource type that you want the Kubernetes Operator to watch, ensure that you set scope to a distinct subset of namespaces.

Follow the relevant [installation instructions](https://www.mongodb.com/docs/kubernetes/current/tutorial/install-k8s-operator.md#std-label-install-k8s-operator) for `helm`, but specify one or more namespaces in the [operator.watchNamespace](https://www.mongodb.com/docs/kubernetes/current/reference/helm-operator-settings.md#std-label-helm-watch-namespace) parameter for the Kubernetes Operator to watch:

```yaml
# Watch one namespace
helm install mongodb-kubernetes-operator mongodb/mongodb-kubernetes \
   --set operator.watchNamespace='namespace-to-watch' <...>
```

```yaml
# Watch both namespace-a and namespace-b
helm install mongodb-kubernetes-operator mongodb/mongodb-kubernetes \
   --set operator.watchNamespace="namespace-a\,namespace-b"
```

```yaml
# Operator with name `mongodb-kubernetes-operator-qa-envs` will
# watch ns-dev, ns-qa and ns-uat namespaces

helm install mongodb-kubernetes-operator-qa-envs mongodb/mongodb-kubernetes \
   --set operator.watchNamespace="ns-dev\,ns-qa\,ns-uat"
```

```yaml
# Operator with name `mongodb-kubernetes-operator-staging` will
# watch ns-staging and ns-pre-prod
helm install mongodb-kubernetes-operator-staging mongodb/mongodb-kubernetes --set operator.watchNamespace="ns-staging\,ns-pre-prod"
```

When installing the Kubernetes Operator to watch resources in one or more namespaces other than the namespace in which the Kubernetes Operator is deployed:

1. Create the following resources:

   - A [ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole) with access to multiple resources. For the full resource definition, see the [operator roles templates](https://github.com//mongodb/mongodb-kubernetes/tree/1.14.0/helm_chart/templates) example. This is a cluster-scoped resource.

   - Create a [ClusterRoleBinding](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding) to link [ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole) with ServiceAccount. This `clusterRoleBinding` will bind the `clusterRole` that you created  with the ServiceAccount that the Kubernetes Operator is using on the namespace where you install it.

2. Include the [ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole) and [ClusterRoleBinding](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding) in the default configuration files that you apply during the installation.

3. Create local Kubernetes [ServiceAccounts:](https://kubernetes.io/docs/concepts/security/service-accounts/)

   For each namespace, create some or all of the following local Kubernetes ServiceAccounts:

   - If you want to deploy a MongoDB instance in the namespace, use `mongodb-kubernetes-database-pods`.

   - If you want to deploy Ops Manager in the namespace, use `mongodb-kubernetes-appdb` and `mongodb-kubernetes-ops-manager`.

The following example illustrates how the [ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole) and [ClusterRoleBinding](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding) work together in the cluster.

Suppose you create a ServiceAccount in the `mongodb` namespace, and then install the Kubernetes Operator in this namespace. The Kubernetes Operator uses this ServiceAccount.

To set the Kubernetes Operator scope to watch namespaces `ns1` and `ns2`:

1. Obtain [cluster-admin privileges.](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles)

2. Using these privileges, create a cluster-wide, non-namespaced [ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole).

3. Create a [ClusterRoleBinding](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding) in three namespaces: `mongodb`, `ns1` and `ns2`. This [ClusterRoleBinding](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding) will bind the [ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole) to the ServiceAccount in the `mongodb` namespace. The `clusterRoleBinding` will allow the Kubernetes Operator deployed in the `mongodb` namespace to access the resources described in the `clusterRole` of the target namespace, that is, in `mongodb`, `ns1` and `ns2`.

See also [operator.watchNamespace.](https://www.mongodb.com/docs/kubernetes/current/reference/helm-operator-settings.md#std-label-helm-watch-namespace)

#### Operator Uses Cluster-Wide Scope

You can set the scope for the Kubernetes Operator to the Kubernetes cluster. In this case, the Kubernetes Operator watches Ops Manager and `MongoDB` resources in all [namespaces](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) in the Kubernetes cluster.

**Important:**

You can deploy only one instance of the Kubernetes Operator with a cluster-wide scope per Kubernetes cluster.

To set a cluster-wide scope for the Kubernetes Operator, follow the instructions for your preferred installation method.

1. Use the [mongodb-kubernetes.yaml](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/mongodb-kubernetes.yaml) sample YAML (Yet Another Markup Language) file from the [MongoDB Controllers for Kubernetes Operator GitHub repository.](https://github.com//mongodb/mongodb-kubernetes)

2. Set the `spec.template.spec.containers.name.env.name:WATCH_NAMESPACE` in [mongodb-kubernetes.yaml](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/mongodb-kubernetes.yaml) to `"*"`. You must include the double quotation marks (`"`) around the asterisk (`*`) in the YAML (Yet Another Markup Language) file.

   ```sh
   WATCH_NAMESPACE: "*"
   ```

3. In [mongodb-kubernetes.yaml](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/mongodb-kubernetes.yaml), change:

   ```sh
   kind:  Role
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
    name: mongodb-kubernetes-operator
   ```

   to:

   ```sh
   kind: ClusterRole
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
    name: mongodb-kubernetes-operator
   ```

4. Add the following code to the `ClusterRole` that you have just modified:

   ```sh
   - apiGroups:
     - ""
     resources:
     - namespaces
     verbs:
     - list
     - watch
   ```

5. In [mongodb-kubernetes.yaml](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/mongodb-kubernetes.yaml), change:

   ```sh
   kind:  RoleBinding
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
    name: mongodb-kubernetes-operator
    namespace: mongodb
   roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: Role
    name: mongodb-kubernetes-operator
   subjects:
    - kind: ServiceAccount
    name: mongodb-kubernetes-operator
    namespace: mongodb
   ```

   to:

   ```sh
   kind:  ClusterRoleBinding
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
    name: mongodb-kubernetes-operator
    namespace: mongodb
   roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: mongodb-kubernetes-operator
   subjects:
    - kind: ServiceAccount
    name: mongodb-kubernetes-operator
    namespace: mongodb
   ```

6. In the `mongodb-kubernetes.yaml` file, change the `<namespace>` value to the namespace where you want the Kubernetes Operator to deploy resources and apply the YAML (Yet Another Markup Language) fle.

   ```yaml
   ---
   kind: ServiceAccount
   apiVersion: v1
   metadata:
     name: mongodb-kubernetes-appdb
     namespace: <namespace>
   ---
   kind: ServiceAccount
   apiVersion: v1
   metadata:
     name: mongodb-kubernetes-database-pods
     namespace: <namespace>
   ---
   kind: ServiceAccount
   apiVersion: v1
   metadata:
     name: mongodb-kubernetes-ops-manager
     namespace: <namespace>
   ---
   kind: Role
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
     name: mongodb-kubernetes-appdb
     namespace: <namespace>
   rules:
     - apiGroups:
         - ""
       resources:
         - secrets
       verbs:
         - get
     - apiGroups:
         - ""
       resources:
         - pods
       verbs:
         - patch
   ---
   kind: RoleBinding
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
     name: mongodb-kubernetes-appdb
     namespace: <namespace>
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: Role
     name: mongodb-kubernetes-appdb
   subjects:
     - kind: ServiceAccount
       name: mongodb-kubernetes-appdb
       namespace: <namespace>
   ...


   ```

7. Create local Kubernetes [service accounts](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/):

   For each namespace, create some or all of the following local Kubernetes [service accounts](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/):

   - If you want to deploy a MongoDB instance in the namespace, use `mongodb-kubernetes-database-pods`.

   - If you want to deploy Ops Manager in the namespace, use `mongodb-kubernetes-appdb` and `mongodb-kubernetes-ops-manager`.

   Copy and paste the applicable examples and replace the `<namespace>` value with the label that identifies the namespace.

   ```sh
   ---
   kind: ServiceAccount
   apiVersion: v1
   metadata:
    name: mongodb-kubernetes-database-pods
    namespace: <namespace>
   ---
   kind: ServiceAccount
   apiVersion: v1
   metadata:
    name: mongodb-kubernetes-appdb
    namespace: <namespace>
   ---
   kind: ServiceAccount
   apiVersion: v1
   metadata:
    name: mongodb-kubernetes-ops-manager
    namespace: <namespace>
   ```

Before you deploy the Kubernetes Operator, configure the following items:

1. Configure the Kubernetes Operator to watch all namespaces:

   ```sh
   helm install mongodb-kubernetes-operator mongodb/mongodb-kubernetes \
     --set operator.watchNamespace="*"
   ```

2. Create local Kubernetes [service accounts](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/):

   For each namespace, create some or all of the following local Kubernetes [service accounts](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/):

   - If you want to deploy a MongoDB instance in the namespace, use `mongodb-kubernetes-database-pods`.

   - If you want to deploy Ops Manager in the namespace, use `mongodb-kubernetes-appdb` and `mongodb-kubernetes-ops-manager`.

   Copy and paste the applicable examples and replace the `<namespace>` value with the label that identifies the namespace.

   ```sh
   helm template mongodb/mongodb-kubernetes \
     --set operator.namespace=<metadata.namespace> \
     --show-only templates/database-roles.yaml | kubectl apply -f -
   ```

Before you deploy the Kubernetes Operator, configure the following items:

1. Use the [mongodb-kubernetes-openshift.yaml](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/mongodb-kubernetes-openshift.yaml) sample YAML (Yet Another Markup Language) file from the [MongoDB Controllers for Kubernetes Operator GitHub repository.](https://github.com//mongodb/mongodb-kubernetes)

2. Set the `spec.template.spec.containers.name.env.name:WATCH_NAMESPACE` in [mongodb-kubernetes-openshift.yaml](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/mongodb-kubernetes-openshift.yaml) to `"*"`. You must include the double quotation marks (`"`) around the asterisk (`*`) in the YAML (Yet Another Markup Language) file.

   ```sh
   WATCH_NAMESPACE: "*"
   ```

3. Create the corresponding roles for these accounts. In [mongodb-kubernetes-openshift.yaml](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/mongodb-kubernetes-openshift.yaml), change:

   ```sh
   kind: Role
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
    name: enterprise-operator
   ```

   to:

   ```sh
   kind: ClusterRole
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
    name: enterprise-operator
   ```

4. Add the following code to the `ClusterRole` that you have just modified:

   ```sh
   - apiGroups:
     - ""
     resources:
     - namespaces
     verbs:
     - list
     - watch
   ```

5. In [mongodb-kubernetes-openshift.yaml](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/mongodb-kubernetes-openshift.yaml), change:

   ```sh
   kind:  RoleBinding
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
    name: enterprise-operator
    namespace: mongodb
   roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: Role
    name: enterprise-operator
   subjects:
    - kind: ServiceAccount
    name: enterprise-operator
    namespace: mongodb
   ```

   to:

   ```sh
   kind:  ClusterRoleBinding
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
    name: enterprise-operator
    namespace: mongodb
   roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: enterprise-operator
   subjects:
    - kind: ServiceAccount
    name: enterprise-operator
    namespace: mongodb
   ```

6. Create the secret only in the namespace where you will deploy the Kubernetes Operator. If you deploy MongoDB resources in [multiple namespaces](https://www.mongodb.com/docs/kubernetes/current/tutorial/plan-k8s-op-considerations.md#std-label-ns-scope-different-ns) or with a [cluster-wide scope](https://www.mongodb.com/docs/kubernetes/current/tutorial/plan-k8s-op-considerations.md#std-label-cluster-wide-scope), the Kubernetes Operator synchronizes the secret across all watched namespaces. To learn more, see the `registry.imagePullSecrets` setting in the [Helm installation settings.](https://www.mongodb.com/docs/kubernetes/current/reference/helm-operator-settings.md#std-label-meko-op-install-settings-helm)

   In the `mongodb-kubernetes.yaml` file, replace `<namespace>` with the namespace in which you want to install the Kubernetes Operator. Use [oc](https://docs.openshift.com/container-platform/4.12/cli_reference/index.html) or the OpenShift Container Platform UI to apply the resulting YAML (Yet Another Markup Language) file.

   ```yaml
   ---
   kind: ServiceAccount
   apiVersion: v1
   metadata:
     name: mongodb-kubernetes-appdb
     namespace: <namespace>
   ---
   kind: ServiceAccount
   apiVersion: v1
   metadata:
     name: mongodb-kubernetes-database-pods
     namespace: <namespace>
   ---
   kind: ServiceAccount
   apiVersion: v1
   metadata:
     name: mongodb-kubernetes-ops-manager
     namespace: <namespace>
   ---
   kind: Role
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
     name: mongodb-kubernetes-appdb
     namespace: <namespace>
   rules:
     - apiGroups:
         - ""
       resources:
         - secrets
       verbs:
         - get
     - apiGroups:
         - ""
       resources:
         - pods
       verbs:
         - patch
   ---
   kind: RoleBinding
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
     name: mongodb-kubernetes-appdb
     namespace: <namespace>
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: Role
     name: mongodb-kubernetes-appdb
   subjects:
     - kind: ServiceAccount
       name: mongodb-kubernetes-appdb
       namespace: <namespace>
   ...


   ```

7. Create local Kubernetes [service accounts](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/):

   For each namespace, create some or all of the following local Kubernetes [service accounts](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/):

   - If you want to deploy a MongoDB instance in the namespace, use `mongodb-kubernetes-database-pods`.

   - If you want to deploy Ops Manager in the namespace, use `mongodb-kubernetes-appdb` and `mongodb-kubernetes-ops-manager`.

   Copy and paste the applicable examples and replace the `<namespace>` value with the label that identifies the namespace.

Before you deploy the Kubernetes Operator, configure the following items:

1. Configure the Kubernetes Operator to watch all namespaces:

   ```sh
   helm install mongodb-kubernetes-operator mongodb/mongodb-kubernetes \
     --set operator.watchNamespace="*" \
   ```

2. Create the secret only in the namespace where you will deploy the Kubernetes Operator. If you deploy MongoDB resources in [multiple namespaces](https://www.mongodb.com/docs/kubernetes/current/tutorial/plan-k8s-op-considerations.md#std-label-ns-scope-different-ns) or with a [cluster-wide scope](https://www.mongodb.com/docs/kubernetes/current/tutorial/plan-k8s-op-considerations.md#std-label-cluster-wide-scope), the Kubernetes Operator synchronizes the secret across all watched namespaces. To learn more, see the `registry.imagePullSecrets` setting in the [Helm installation settings.](https://www.mongodb.com/docs/kubernetes/current/reference/helm-operator-settings.md)

   In the `mongodb-kubernetes.yaml` file, replace `<namespace>` with the namespace in which you want to install the Kubernetes Operator. Use [oc](https://docs.openshift.com/container-platform/4.12/cli_reference/index.html) or the OpenShift Container Platform UI to apply the resulting YAML (Yet Another Markup Language) file.

   ```yaml
   ---
   kind: ServiceAccount
   apiVersion: v1
   metadata:
     name: mongodb-kubernetes-appdb
     namespace: <namespace>
   ---
   kind: ServiceAccount
   apiVersion: v1
   metadata:
     name: mongodb-kubernetes-database-pods
     namespace: <namespace>
   ---
   kind: ServiceAccount
   apiVersion: v1
   metadata:
     name: mongodb-kubernetes-ops-manager
     namespace: <namespace>
   ---
   kind: Role
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
     name: mongodb-kubernetes-appdb
     namespace: <namespace>
   rules:
     - apiGroups:
         - ""
       resources:
         - secrets
       verbs:
         - get
     - apiGroups:
         - ""
       resources:
         - pods
       verbs:
         - patch
   ---
   kind: RoleBinding
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
     name: mongodb-kubernetes-appdb
     namespace: <namespace>
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: Role
     name: mongodb-kubernetes-appdb
   subjects:
     - kind: ServiceAccount
       name: mongodb-kubernetes-appdb
       namespace: <namespace>
   ...


   ```

3. Create local Kubernetes [service accounts](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/):

   For each namespace, create some or all of the following local Kubernetes [service accounts](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/):

   - If you want to deploy a MongoDB instance in the namespace, use `mongodb-kubernetes-database-pods`.

   - If you want to deploy Ops Manager in the namespace, use `mongodb-kubernetes-appdb` and `mongodb-kubernetes-ops-manager`.

   Copy and paste the applicable examples and replace the `<namespace>` value with the label that identifies the namespace.

   ```sh
   helm template mongodb/mongodb-kubernetes \
     --set operator.namespace=<metadata.namespace> \
     --show-only templates/database-roles.yaml | oc apply -f -
   ```

## Specify CPU and Memory Resource Requirements

**Note:**

The following considerations apply:

- All sizing and performance recommendations for common MongoDB deployments through the Kubernetes Operator in this section are subject to change. Do not treat these recommendations as guarantees or limitations of any kind.

- These recommendations reflect performance testing findings and represent our suggestions for production deployments. We ran the tests on a cluster comprised of seven AWS EC2 instances of type `t2.2xlarge` and a master node of type `t2.medium`.

- The recommendations in this section don't discuss characteristics of any specific deployment. Your deployment's characteristics may differ from the assumptions made to create these recommendations. Contact [MongoDB Support](https://support.mongodb.com/welcome) for further help with sizings.

In Kubernetes, each Pod includes parameters that allow you to specify [CPU resources](https://kubernetes.io/docs/tasks/configure-pod-container/assign-cpu-resource/) and [memory resources](https://kubernetes.io/docs/tasks/configure-pod-container/assign-memory-resource/) for each container in the Pod.

To indicate resource bounds, Kubernetes uses the [requests and limits](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits) parameters, where:

- *request* indicates a lower bound of a resource.

- *limit* indicates an upper bound of a resource.

The following sections illustrate how to:

- [set CPU and Memory for the Operator Pod.](https://www.mongodb.com/docs/kubernetes/current/tutorial/plan-k8s-op-considerations.md#std-label-operator_pod_resources)

- [set CPU and Memory for MongoDB Pods.](https://www.mongodb.com/docs/kubernetes/current/tutorial/plan-k8s-op-considerations.md#std-label-mdb_pods_resources)

For the Pods hosting Ops Manager, use the [default resource limits configurations.](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/samples/ops-manager/ops-manager-pod-spec.yaml#L38-L46)

## Consider Recommended Practices for Storage

The following recommendations are applicable to MongoDB deployments and Ops Manager Application database deployments managed by the Kubernetes Operator.

**Note:**

- Redundancy in the storage class is not required because MongoDB automatically replicates data between the members of a replica set or shard.

- You can't share storage across members of a replica set or shard.

* Use the storage class that provides the best performance given your constraints. See [scale a deployment](https://www.mongodb.com/docs/kubernetes/current/tutorial/scale-resources.md#std-label-scale-k8s-resources) to learn more.

* Provision [Persistent Volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) that support `ReadWriteOnce` for your storage class.

* On a worker node level, apply the best practices from [MongoDB on Linux.](https://www.mongodb.com/docs/manual/administration/production-notes.md#std-label-production-notes)

* Ensure that you use a storage class that supports [volume expansion](https://kubernetes.io/docs/concepts/storage/storage-classes/#allow-volume-expansion) to enable database volume resizing.

## Use Static Containers (Public Preview)

Static containers are simpler and more secure than non-static containers. Static containers are immutable at runtime, which means that they don't change from the image used to create the container. In addition:

- While running, static containers don't download binaries or run scripts or other utilities over network connections. Static containers only download runtime configuration files.

- While running, static containers don't modify any file except storage volume mounts.

- You can run security scans on the container images to determine what is actually run as a live container, and the container that runs won't run binaries other than what was defined in the image.

- Static containers don't require that you host the MongoDB binary on either Ops Manager or another HTTPS (Hypertext Transfer Protocol Secure) server, which is especially useful if you have an air-gapped environment.

- You can't run extensive `CMD` scripts for the static container.

- You can't copy files between static containers using `initContainer`.

**Note:**

When using static containers, the container layout and resource limits differ between MongoDB Database CR and Application Database deployments.

When deployed as static containers, a MongoDB Database CR consists of two containers with distinct roles:

- The `mongodb-agent` container runs the MongoDB Agent, which then runs the `mongod` process. This container handles the actual database workload.

- The `mongodb-enterprise-server` container provides the MongoDB binaries but does not run any active processes.

To modify the resource limits for the MongoDB database resource, specify your desired resource limits on the `mongodb-agent` container. Resource limits set on the `mongodb-enterprise-server` container have no functional impact on performance, though they still count toward node resource allocation.

For the Application Database in static architecture, `mongod` runs directly in the `mongod` container, not in `mongodb-agent`. To modify the resource limits for the Application Database, specify your desired resource limits on the `mongod` container instead.

To learn more, see [Static Containers (Public Preview).](https://www.mongodb.com/docs/kubernetes/current/tutorial/plan-k8s-op-container-images.md#std-label-static-containers)

## Co-locate `mongos` Pods with Your Applications

You can run the lightweight `mongos` instance on the same [node](https://kubernetes.io/docs/concepts/architecture/nodes/) as your apps using MongoDB. The Kubernetes Operator supports standard Kubernetes [node affinity and anti-affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity) features. Using these features, you can force install the `mongos` on the same node as your application.

The following abbreviated example shows affinity and multiple availability zones configuration.

The `podAffinity` key determines whether to install an application on the same Pod, node, or data center as another application.

To specify Pod affinity:

1. Add a label and value in the `spec.podSpec.podTemplate.metadata.labels` YAML (Yet Another Markup Language) collection to tag the deployment. See [`spec.podSpec.podTemplate.metadata`](https://www.mongodb.com/docs/kubernetes/current/reference/k8s-operator-specification.md#mongodb-setting-spec.podSpec.podTemplate.metadata), and the [Kubernetes PodSpec v1 core API.](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#podspec-v1-core)

2. Specify which label the `mongos` uses in the `spec.mongosPodSpec.podAffinity` `.requiredDuringSchedulingIgnoredDuringExecution.labelSelector` YAML (Yet Another Markup Language) collection. The `matchExpressions` collection defines the `label` that the Kubernetes Operator uses to identify the Pod for hosting the `mongos`.

**Example:**

```yaml
apiVersion: mongodb.com/v1
kind: MongoDB
metadata:
  name: my-replica-set
spec:
  members: 3
  version: 8.0.0
  service: my-service

  ...
    podTemplate:
      spec:
        affinity:
          podAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              - labelSelector:
                  matchExpressions:
                    - key: security
                      operator: In
                      values:
                        - S1
                topologyKey: failure-domain.beta.kubernetes.io/zone
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              nodeSelectorTerms:
                - matchExpressions:
                    - key: kubernetes.io/e2e-az-name
                      operator: In
                      values:
                        - e2e-az1
                        - e2e-az2
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - podAffinityTerm:
                topologyKey: nodeId
```

See the full example of multiple availability zones and node affinity configuration in [replica-set-affinity.yaml](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/samples/mongodb/affinity/replica-set-affinity.yaml) in the [Affinity Samples](https://github.com//mongodb/mongodb-kubernetes/tree/1.14.0/public/samples/mongodb/persistent-volumes) directory.

This directory also contains sample affinity and multiple zones configurations for sharded clusters and standalone MongoDB deployments.

**See also:**

- [Assigning Pods to Nodes](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector)

- [Node affinity and anti-affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity)

- [Kubernetes PodSpec v1 core API](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#podspec-v1-core)

## Name Your MongoDB Service with its Purpose

Set the [`spec.service`](https://www.mongodb.com/docs/kubernetes/current/reference/k8s-operator-specification.md#mongodb-setting-spec.service) parameter to a value that identifies this deployment's purpose, as illustrated in the following example.

```yaml
apiVersion: mongodb.com/v1
kind: MongoDB
metadata:
  name: my-replica-set
spec:
  members: 3
  version: "8.0.0"
  service: drilling-pumps-geosensors
  featureCompatibilityVersion: "8.0"
```

**See also:**

[`spec.service`](https://www.mongodb.com/docs/kubernetes/current/reference/k8s-operator-specification.md#mongodb-setting-spec.service)

## Use Labels to Differentiate Between Deployments

Use the [Pod affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#inter-pod-affinity-and-anti-affinity) Kubernetes feature to:

- Separate different MongoDB resources, such as `test`, `staging`, and `production` environments.

- Place [Pods](https://kubernetes.io/docs/concepts/workloads/pods/pod/) on some specific nodes to take advantage of features such as SSD (Solid State Disk) support.

```yaml
mongosPodSpec:
  podAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
        matchExpressions:
        - key: security
          operator: In
          values:
          - S1
        topologyKey: failure-domain.beta.kubernetes.io/zone
```

**See also:**

[Pod affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#inter-pod-affinity-and-anti-affinity)

## Customize the CustomResourceDefinitions that the Kubernetes Operator Watches

You can specify which custom resources you want the Kubernetes Operator to watch. This allows you to install the CustomResourceDefinition for only the resources that you want the Kubernetes Operator to manage.

You must use `helm` to configure the Kubernetes Operator to watch only the custom resources you specify. Follow the relevant `helm` [installation instructions](https://www.mongodb.com/docs/kubernetes/current/tutorial/install-k8s-operator.md#std-label-install-k8s-operator), but make the following adjustments:

1. Decide which CustomResourceDefinitions you want to install. You can install any number of the following:

   | Value | Description |
   | --- | --- |
   | `mongodb` | Install the CustomResourceDefinitions for database resources and watch those resources. |
   | `mongodbusers` | Install the CustomResourceDefinitions for MongoDB user resources and watch those resources. |
   | `opsmanagers` | Install the CustomResourceDefinitions for Ops Manager resources and watch those resources. |

2. Install the Helm Chart and specify which CustomResourceDefinitions you want the Kubernetes Operator to watch.

   Separate each custom resource with a comma:

   ```sh
   helm install <deployment-name> mongodb/mongodb-kubernetes \
     --set operator.watchedResources="{mongodb,mongodbusers}" \
        --skip-crds
   ```

## Ensure Proper Persistence Configuration

The Kubernetes deployments orchestrated by the Kubernetes Operator are stateful. The Kubernetes container uses [Persistent Volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) to maintain the cluster state between restarts.

To satisfy the statefulness requirement, the Kubernetes Operator performs the following actions:

- Creates [Persistent Volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) for your MongoDB deployment.

- Mounts storage devices to one or more directories called mount points.

- Creates one persistent volume for each MongoDB mount point.

- Sets the default path in each Kubernetes container to `/data`.

To meet your MongoDB cluster's storage needs, make the following changes in your configuration for each replica set deployed with the Kubernetes Operator:

- Verify that persistent volumes are enabled in [`spec.persistent`](https://www.mongodb.com/docs/kubernetes/current/reference/k8s-operator-specification.md#mongodb-setting-spec.persistent). This setting defaults to `true`.

- Specify a sufficient amount of storage for the Kubernetes Operator to allocate for each of the volumes. The volumes store the data and the logs.

  - To set multiple volumes, each for data, logs, and the `oplog`, use [`spec.podSpec.persistence.multiple.data`.](https://www.mongodb.com/docs/kubernetes/current/reference/k8s-operator-specification.md#mongodb-setting-spec.podSpec.persistence.multiple.data)

  - To set a single volume to store data, logs, and the `oplog`, use [`spec.podSpec.persistence.single`.](https://www.mongodb.com/docs/kubernetes/current/reference/k8s-operator-specification.md#mongodb-setting-spec.podSpec.persistence.single)

The following abbreviated example shows recommended persistent storage sizes.

```yaml
apiVersion: mongodb.com/v1
kind: MongoDB
metadata:
  name: my-replica-cluster
spec:

  ...
  persistent: true


  shardPodSpec:
  ...
    persistence:
      multiple:
        data:
          storage: "20Gi"
        logs:
          storage: "4Gi"
          storageClass: standard
```

For a full example of persistent volumes configuration, see [replica-set-persistent-volumes.yaml](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/samples/mongodb/persistent-volumes/replica-set-persistent-volumes.yaml) in the [Persistent Volumes Samples](https://github.com//mongodb/mongodb-kubernetes/tree/1.14.0/public/samples/mongodb/persistent-volumes) directory. This directory also contains sample persistent volumes configurations for sharded clusters and standalone deployments.

**See also:**

- [`spec.persistent`](https://www.mongodb.com/docs/kubernetes/current/reference/k8s-operator-specification.md#mongodb-setting-spec.persistent)

- [`spec.podSpec.persistence.single`](https://www.mongodb.com/docs/kubernetes/current/reference/k8s-operator-specification.md#mongodb-setting-spec.podSpec.persistence.single)

- [`spec.podSpec.persistence.multiple.data`](https://www.mongodb.com/docs/kubernetes/current/reference/k8s-operator-specification.md#mongodb-setting-spec.podSpec.persistence.multiple.data)

## Set CPU and Memory Utilization Bounds for the Kubernetes Operator Pod

When you deploy MongoDB replica sets with the Kubernetes Operator, the initial reconcilliation process increases CPU usage for the Pod running the Kubernetes Operator. However, when the replica set deployment process completes, the CPU usage by the Kubernetes Operator reduces considerably.

**Note:**

The severity of CPU usage spikes in the Kubernetes Operator is directly impacted by [the thread count](https://www.mongodb.com/docs/kubernetes/current/tutorial/plan-k8s-op-considerations.md#std-label-increase-thread-count-ops-manager) of the Kubernetes Operator, as the thread count (defined by the [MDB\_MAX\_CONCURRENT\_RECONCILES](https://www.mongodb.com/docs/kubernetes/current/reference/kubectl-operator-settings.md#std-label-mdb-max-concurrent-reconciles) value) is equal to the number of reconcilliation processes that can be running in parallel at any given time.

For production deployments, to satisfy deploying up to 50 MongoDB replica sets or sharded clusters in parallel with the Kubernetes Operator, set the CPU and memory resources and limits for the Kubernetes Operator Pod as follows:

- `spec.template.spec.containers.resources.requests.cpu` to 500m

- `spec.template.spec.containers.resources.limits.cpu` to 1100m

- `spec.template.spec.containers.resources.requests.memory` to 200Mi

- `spec.template.spec.containers.resources.limits.memory` to 1Gi

If you use Helm to deploy resources, define these values in the [values.yaml file.](https://www.mongodb.com/docs/kubernetes/current/reference/helm-operator-settings.md#std-label-k8s-op-resources-setting)

The following abbreviated example shows the configuration with recommended CPU and memory bounds for the Kubernetes Operator Pod in your deployment of 50 replica sets or sharded clusters. If you are deploying fewer than 50 MongoDB clusters, you may use lower numbers in the configuration file for the Kubernetes Operator Pod.

**Example:**

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
 name: mongodb-kubernetes-operator
 namespace: mongodb
spec:
 replicas: 1
 selector:
  matchLabels:
     app.kubernetes.io/component: controller
     app.kubernetes.io/name: mongodb-kubernetes-operator
     app.kubernetes.io/instance: mongodb-kubernetes-operator
 template:
  metadata:
   labels:
     app.kubernetes.io/component: controller
     app.kubernetes.io/name: mongodb-kubernetes-operator
     app.kubernetes.io/instance: mongodb-kubernetes-operator
   spec:
     serviceAccountName: mongodb-kubernetes-operator
     securityContext:
       runAsNonRoot: true
       runAsUser: 2000
     containers:
     - name: mongodb-kubernetes-operator
       image: quay.io/mongodb/mongodb-kubernetes-operator:1.9.2
       imagePullPolicy: Always
       args:
        - "-watch-resource=mongodb"
        - "-watch-resource=opsmanagers"
        - "-watch-resource=mongodbusers"
       command:
        - "/usr/local/bin/mongodb-kubernetes-operator"
       resources:
         limits:
           cpu: 1100m
           memory: 1Gi
         requests:
           cpu: 500m
           memory: 200Mi
```

For a full example of CPU and memory utilization resources and limits for the Kubernetes Operator Pod that satisfy parallel deployment of up to 50 MongoDB replica sets, see the [mongodb-kubernetes.yaml](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/mongodb-kubernetes.yaml#L372-L378) file.

**See also:**

- [Requests and Limits](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits)

- [Assign CPU Resources to Containers and Pods](https://kubernetes.io/docs/tasks/configure-pod-container/assign-cpu-resource/)

## Set CPU and Memory Utilization Bounds for MongoDB Pods

The values for Pods hosting replica sets or sharded clusters map to the [requests field](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#resourcerequirements-v1-core) for CPU and memory for the created Pod. These values are consistent with [considerations](https://www.mongodb.com/docs/manual/administration/production-notes.md#std-label-prod-notes-ram) stated for MongoDB hosts.

The Kubernetes Operator uses its allocated memory for processing, for the WiredTiger cache, and for storing packages during the deployments.

For production deployments, set the CPU and memory resources and limits for the MongoDB Pod as follows:

- `spec.podSpec.podTemplate.spec.containers.resources.requests.cpu` to 0.25

- `spec.podSpec.podTemplate.spec.containers.resources.limits.cpu` to 0.25

- `spec.podSpec.podTemplate.spec.containers.resources.requests.memory` to 512M

- `spec.podSpec.podTemplate.spec.containers.resources.limits.memory` to 512M

If you use Helm to deploy resources, define these values in the [values.yaml file.](https://www.mongodb.com/docs/kubernetes/current/reference/helm-operator-settings.md#std-label-k8s-op-resources-setting)

The following abbreviated example shows the configuration with recommended CPU and memory bounds for each Pod hosting a MongoDB replica set member in your deployment.

**Example:**

```yaml
apiVersion: mongodb.com/v1
kind: MongoDB
metadata:
name: my-replica-set
spec:
  members: 3
  version: 8.0.0
  service: my-service
  ...

  persistent: true
  podSpec:
    podTemplate:
      spec:
        containers:
        - name: mongodb-enterprise-database
          resources:
            limits:
              cpu: "0.25"
              memory: 512M
```

For a full example of CPU and memory utilization resources and limits for Pods hosting MongoDB replica set members, see the [replica-set-podspec.yaml](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/samples/mongodb/podspec/replica-set-podspec.yaml#L38-L45) file in the [MongoDB Podspec Samples](https://github.com//mongodb/mongodb-kubernetes/tree/1.14.0/public/samples/mongodb/podspec) directory.

This directory also contains sample CPU and memory limits configurations for Pods used for:

- A sharded cluster, in the [sharded-cluster-podspec.yaml.](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/samples/mongodb/podspec/sharded-cluster-podspec.yaml#L31-L91)

- Standalone MongoDB deployments, in the [standalone-podspec.yaml.](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/samples/mongodb/podspec/standalone-podspec.yaml#L36-39)

**See also:**

- [`spec.podSpec.podTemplate.spec`](https://www.mongodb.com/docs/kubernetes/current/reference/k8s-operator-specification.md#mongodb-setting-spec.podSpec.podTemplate.spec)

- [Requests and Limits](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits)

- [Assign CPU Resources to Containers and Pods](https://kubernetes.io/docs/tasks/configure-pod-container/assign-cpu-resource/)

## Use Multiple Availability Zones

Set the Kubernetes Operator and [statefulSets](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) to distribute all members of one replica set to different [nodes](https://kubernetes.io/docs/concepts/architecture/nodes/) to ensure high availability.

The following abbreviated example shows affinity and multiple availability zones configuration.

**Example:**

```yaml
apiVersion: mongodb.com/v1
kind: MongoDB
metadata:
  name: my-replica-set
spec:
  members: 3
  version: 8.0.0
  service: my-service
  ...
    podAntiAffinityTopologyKey: nodeId
    podAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
        matchExpressions:
        - key: security
          operator: In
          values:
          - S1
        topologyKey: failure-domain.beta.kubernetes.io/zone

    nodeAffinity:
       requiredDuringSchedulingIgnoredDuringExecution:
         nodeSelectorTerms:
         - matchExpressions:
           - key: kubernetes.io/e2e-az-name
           operator: In
           values:
           - e2e-az1
           - e2e-az2
```

In this example, the Kubernetes Operator schedules the Pods deployment to the nodes which have the label `kubernetes.io/e2e-az-name` in `e2e-az1` or `e2e-az2` availability zones. Change `nodeAffinity` to schedule the deployment of Pods to the desired availability zones.

See the full example of multiple availability zones configuration in [replica-set-affinity.yaml](https://github.com//mongodb/mongodb-kubernetes/blob/1.14.0/public/samples/mongodb/affinity/replica-set-affinity.yaml) in the [Affinity Samples](https://github.com//mongodb/mongodb-kubernetes/tree/1.14.0/public/samples/mongodb/persistent-volumes) directory.

This directory also contains sample affinity and multiple zones configurations for sharded clusters and standalone MongoDB deployments.

**See also:**

- [Running in Multiple Zones](https://kubernetes.io/docs/setup/best-practices/multiple-zones/)

- [Node affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity)

## Increase Thread Count to Run multiple Reconciliation Processes in Parallel

If you plan to deploy more than 10 MongoDB replica sets in parallel, you can configure the Kubernetes Operator to run multiple reconciliation processes in parallel by setting [MDB\_MAX\_CONCURRENT\_RECONCILES](https://www.mongodb.com/docs/kubernetes/current/reference/kubectl-operator-settings.md#std-label-mdb-max-concurrent-reconciles) environment variable in your Kubernetes Operator deployment or or through the [operator.maxConcurrentReconciles](https://www.mongodb.com/docs/kubernetes/current/reference/helm-operator-settings.md#std-label-mdb-max-concurrent-reconciles-helm) field in your Helm `values.yaml` file to configure a higher thread count.

Increasing the thread count of the Kubernetes Operator allows you to vertically scale your Kubernetes Operator deployment to hundreds of `MongoDB` resources running within your Kubernetes cluster and optimize CPU utilization.

Please monitor Kubernetes API server and Kubernetes Operator resource usage and adjust their respective resource allocation if necessary.

**Note:**

- Proceed with caution when increasing the [MDB\_MAX\_CONCURRENT\_RECONCILES](https://www.mongodb.com/docs/kubernetes/current/reference/kubectl-operator-settings.md#std-label-mdb-max-concurrent-reconciles) beyond 10. In particular, you must monitor the Kubernetes Operator, and the Kubernetes API closely to avoid downtime resulting from increased load on those components.

  To determine the thread count that suits your deployment's needs, use the following guidelines:

  - Your requirements for how responsive the Kubernetes Operator must be when reconciling many resources

  - The compute resources available within your Kubernetes environment and the total processing load your Kubernetes compute resources are under, including resources that may be unrelated to MongoDB

- An alternative to increasing the thread count of a single Kubernetes Operator instance, while still increasing the number of `MongoDB` resources you can support in your Kubernetes cluster, is to deploy multiple Kubernetes Operator instances within your Kubernetes cluster. However, deploying multiple Kubernetes Operator instances requires that you ensure that no two Kubernetes Operator instances are monitoring the same `MongoDB` resources.

  Running more than one instance of the Kubernetes Operator should be done with care, as more Kubernetes Operator instances (especially with parallel reconciliation enabled) put the API server at greater risk of being overwhelmed.

- Scaling of the Kubernetes API server is not a valid reason to run more than one instance of the Kubernetes Operator. If you observe that performance of the API server is affected, adding more instances of the Kubernetes Operator is likely to compound the problem.