Deploy Replica Sets in a Multi-Kubernetes Cluster without a Service Mesh
Use this procedure to deploy a replica set in a multi-Kubernetes cluster MongoDB deployment without using a service mesh for establishing external connectivity between member Kubernetes clusters.
As an alternative to using this procedure, you can use the Multi-Kubernetes-Cluster Quick Start, which uses a service mesh.
Before You Begin
Learn about multi-Kubernetes-cluster deployments.
Review the list of multi-Kubernetes-cluster services and tools.
Complete the prerequisites, but don't set up a service mesh. Instead, decide whether you need a service mesh. If you choose to deploy without a service mesh, use external domains and configure DNS to enable external connectivity.
As part of completing the prerequisites, generate valid certificates for TLS encryption. To learn more, see Prepare for TLS-Encrypted Connections.
Deploy the Ops Manager resources.
Install the Kubernetes Operator in a multi-Kubernetes cluster MongoDB deployment. See Multi-Kubernetes-Cluster Quick Start.
Overview
In a multi-Kubernetes cluster MongoDB deployment without a service mesh, use the following
MongoDBMultiCluster
resource settings:
Use the spec.clusterSpecList.externalAccess.externalService setting so that the Kubernetes Operator creates an external service, and as part of its default configuration, configures a load balancer with default settings. Configure the load balancer to serve as the TCP proxy with a TLS passthrough (no TLS termination in the load balancer).
Customize external services that the Kubernetes Operator creates for each Pod in the Kubernetes cluster. Use the global "all-clusters" configuration in the spec.externalAccess settings and Kubernetes cluster-specific overrides in the spec.clusterSpecList.externalAccess.externalService settings.
Specify cloud provider-specific annotations for the load balancer in the spec.clusterSpecList.externalAccess.externalService.annotations for each Kubernetes cluster.
Specify an external domain in spec.clusterSpecList.externalAccess.externalDomain. This allows the Kubernetes Operator to register
mongod
processes by using the domain suffix. This enables external connections to themongod
processes in a multi-Kubernetes cluster MongoDB deployment.
Deploy a MongoDBMultiCluster
Resource without a Service Mesh
The following procedure establishes TLS-encrypted connections between MongoDB hosts in a replica set, and between client applications and MongoDB deployments.
Create the secret for the TLS certificate of your MongoDBMultiCluster
resource.
Run the kubectl
command to create a new secret that stores the
MongoDBMultiCluster
resource certificate:
kubectl --context $MDB_CENTRAL_CLUSTER_FULL_NAME \ --namespace=<metadata.namespace> \ create secret tls <prefix>-<metadata.name>-cert \ --cert=<resource-tls-cert> \ --key=<resource-tls-key>
Note
You must prefix your secrets with <prefix>-<metadata.name>
.
For example, if you call your deployment my-deployment
and you set
the prefix to mdb
, you must name the TLS secret for the
client TLS communications mdb-my-deployment-cert
. Also,
you must name the TLS secret for internal cluster authentication
(if enabled) mdb-my-deployment-clusterfile
.
Create the ConfigMap to link your CA with your MongoDBMultiCluster
resource.
Run the kubectl
command to link your CA to your MongoDBMultiCluster
resource.
Specify the CA certificate file that you must always name
ca-pem
for the MongoDBMultiCluster
resource:
kubectl --context $MDB_CENTRAL_CLUSTER_FULL_NAME \ --namespace=<metadata.namespace> \ create configmap custom-ca -from-file=ca-pem=<your-custom-ca-file>
Configure kubectl
to use the central cluster's namespace.
If you have not done so already, run the following commands to run
all kubectl
commands on the central cluster in the default
namespace.
kubectl config use-context $MDB_CENTRAL_CLUSTER_FULL_NAME kubectl config set-context $(kubectl config current-context) \ --namespace=mongodb
Copy and paste the sample resource.
Copy the sample replica set YAML file and paste it into a new text file.
Change the file's settings to match your desired replica set configuration.
1 # Provides statefulSet override per cluster 2 3 apiVersion: mongodb.com/v1 4 kind: MongoDBMultiCluster 5 metadata: 6 name: multi-replica-set 7 spec: 8 version: 6.0.0-ent 9 type: ReplicaSet 10 credentials: my-credentials 11 opsManager: 12 configMapRef: 13 name: my-project 14 externalAccess: 15 externalService: 16 annotations: 17 # Global cloud-specific annotations added to external services in all clusters 18 spec: 19 # ServiceSpec attributes to override in external services in all clusters 20 clusterSpecList: 21 - clusterName: cluster1.example.com 22 members: 2 23 externalAccess: 24 # Domain suffix that mongod processes will use in cluster1 25 externalDomain: cluster1.example.com 26 externalService: 27 annotations: 28 # Cloud-specific annotations for external services 29 spec: 30 # ServiceSpec attributes to override if necessary 31 - clusterName: cluster2.example.com 32 members: 1 33 externalAccess: 34 # Domain suffix that mongod processes will use in cluster2 35 externalDomain: cluster2.example.com 36 externalService: 37 annotations: 38 # Cloud-specific annotations for external services 39 spec: 40 # ServiceSpec attributes to override if necessary 41 - clusterName: cluster3.example.com 42 members: 1 43 externalAccess: 44 # Domain suffix that mongod processes will use in cluster3 45 externalDomain: cluster3.example.com 46 externalService: 47 annotations: 48 # Cloud-specific annotations for external services 49 spec: 50 # ServiceSpec attributes to override if necessary 51 52 ...
Define external connectivity settings.
Specify global values that affect all clusters in a multi-Kubernetes cluster MongoDB deployment using the spec.externalAccess settings and cluster-specific overrides using the spec.clusterSpecList.externalAccess.externalService settings.
When you provide these settings in the MongoDBMultiCluster
resource specification,
the Kubernetes Operator creates external services for each Pod in all
Kubernetes clusters. You then use these services to establish external
connectivity to all mongod
processes in your deployment.
Define an external domain for each Kubernetes member cluster.
Define an external domain for each member cluster using the spec.clusterSpecList.externalAccess.externalDomain setting.
As a result, the Kubernetes Operator registers all mongod
processes in the
Kubernetes member cluster under a hostname according to the following convention:
<pod-name>.<externalDomain>
For example, a mongod
process may have the following hostname:
my-replica-set-0-0.cluster-1.example.com
.
Change the settings to your preferred values.
Key | Type | Description | Example |
---|---|---|---|
string | Label for the Resource names must be 44 characters or less. See also | multi-replica-set | |
string | Version of MongoDB that this The format should be ImportantEnsure that you choose a compatible MongoDB Server version. Compatible versions differ depending on the base image that the MongoDB database resource uses. To learn more about MongoDB versioning, see MongoDB Versioning in the MongoDB Manual. | 6.0.0-ent | |
string | Name of the ConfigMap with the Ops Manager connection
configuration. The
This value must exist on the same namespace as the resource you want to create. ImportantOperator manages changes to the ConfigMapThe Kubernetes Operator tracks any changes to the ConfigMap and
reconciles the state of the | <my-project> | |
spec .clusterSpecList .clusterName | string | Name of the cluster in the MongoDBMultiCluster resource. | cluster1.example.com |
spec .clusterSpecList .members | integer | The number of members in this cluster. | 2 |
spec .clusterSpecList .statefulSet .spec | collection | Optional. Provides the configuration for the StatefulSet override for each of
the cluster's StatefulSets in a multi-Kubernetes cluster MongoDB deployment. If specified at an individual
cluster level under | See the example. |
spec .clusterSpecList .statefulSet .spec .volumeClaimTemplates .spec | collection | Optional. If specified, provides a per-cluster override for the default
storage size of the volumeClaimtemplates, for the persistent volume that stores the data. | See the example. |
string | Name of the secret you created as Ops Manager API authentication credentials for the Kubernetes Operator to communicate with Ops Manager. The Ops Manager Kubernetes Secret object holding the Credentials must exist on the same Namespace as the resource you want to create. ImportantOperator manages changes to the SecretThe Kubernetes Operator tracks any changes to the Secret and
reconciles the state of the | <mycredentials> | |
string | Type of MongoDB resource to create. The only supported value for this
field is ReplicaSet . See Limitations. | ReplicaSet |
Add any additional accepted settings for a MongoDBMultiCluster
resource deployment.
You can also add any optional settings to the object specification. See Multi-Kubernetes-Cluster Resource Specification.
Save this replica set config file with a .yaml
extension.
Start your replica set deployment.
In any directory, invoke the following Kubernetes command to create your replica set:
kubectl apply -f <replica-set-conf>.yaml
Verify external connectivity for each member cluster.
Check the status of external services in all member clusters:
kubectl get services Kubernetes should return one external service created for each Pod of the replica set in all member clusters.
Verify that each external service is exposed externally and is reachable. Run the command similar to the following example:
mongosh mongodb://my-replica-set-0-0.cluster-0.example.com:27017 \ -tls -tlsCAFile "issuer-ca.pem" Connecting to
my-replica-set-0-0.cluster-0.example.com:27017
should direct client traffic to an external service namedmy-replica-set-0-0-svc-external
, which, in turn, directs traffic to themongod
process.Configure your DNS zone for the specified external domain to point to the corresponding external services. This configuration depends on your environment or the cloud provider you are using.
Track the status of your multi-Kubernetes cluster MongoDB deployment.
To check the status of your MongoDBMultiCluster
resource, use the following command on the central cluster:
kubectl get mdbmc <resource-name> -o yaml -w
With the -w
(watch) flag set, when the configuration changes, the output
refreshes immediately until the status phase achieves the Running
state.
To learn more about resource deployment statuses, see Troubleshoot the Kubernetes Operator.