Kubernetes is not vendor specific; however, for the sake of providing explicit and complete instructions for deploying all elements of a multi-cluster Kubernetes Operator implementation, this section walks you through the process of deploying multiple Kubernetes clusters on GCP specifically.
Prerequisites
Before you begin, perform the following actions:
Create a GCP account.
Prepare a GCP project so that you can use it to create GKE (Google Kubernetes Engine) clusters.
Install the GCP CLI.
Authenticate with the
gcloud auth logincommand.Update as needed and set the environment variables defined in the following
env_variables.shfile.
1 # GCP project name - this is the only parameter that is mandatory to change here 2 export MDB_GKE_PROJECT="${MDB_GKE_PROJECT:="### Set your GKE project name here ###"}" 3 4 # Adjust the values for each Kubernetes cluster in your deployment. 5 # The deployment script references the following variables to get values for each cluster. 6 export K8S_CLUSTER_0="k8s-mdb-0${K8S_CLUSTER_SUFFIX:-""}" 7 export K8S_CLUSTER_0_ZONE="europe-central2-a" 8 export K8S_CLUSTER_0_NUMBER_OF_NODES=3 9 export K8S_CLUSTER_0_MACHINE_TYPE="e2-standard-4" 10 export K8S_CLUSTER_0_CONTEXT_NAME="gke_${MDB_GKE_PROJECT}_${K8S_CLUSTER_0_ZONE}_${K8S_CLUSTER_0}" 11 12 export K8S_CLUSTER_1="k8s-mdb-1${K8S_CLUSTER_SUFFIX:-""}" 13 export K8S_CLUSTER_1_ZONE="europe-central2-b" 14 export K8S_CLUSTER_1_NUMBER_OF_NODES=3 15 export K8S_CLUSTER_1_MACHINE_TYPE="e2-standard-4" 16 export K8S_CLUSTER_1_CONTEXT_NAME="gke_${MDB_GKE_PROJECT}_${K8S_CLUSTER_1_ZONE}_${K8S_CLUSTER_1}" 17 18 export K8S_CLUSTER_2="k8s-mdb-2${K8S_CLUSTER_SUFFIX:-""}" 19 export K8S_CLUSTER_2_ZONE="europe-central2-c" 20 export K8S_CLUSTER_2_NUMBER_OF_NODES=1 21 export K8S_CLUSTER_2_MACHINE_TYPE="e2-standard-4" 22 export K8S_CLUSTER_2_CONTEXT_NAME="gke_${MDB_GKE_PROJECT}_${K8S_CLUSTER_2_ZONE}_${K8S_CLUSTER_2}" 23 24 # Preemptible nodes disabled — caused random mid-test evictions in CI. 25 # export GKE_SPOT_INSTANCES_SWITCH="--preemptible"
Source Code
You can find all included source code in the MongoDB Kubernetes Operator repository.
Procedure
Create Kubernetes clusters.
Create three GKE (Google Kubernetes Engine) clusters. The Kubernetes Operator is deployed to one of these clusters, and MongoDB custom resources are deployed across all clusters, and are managed by the Kubernetes Operator.
1 gcloud container clusters create "${K8S_CLUSTER_0}" \ 2 --zone="${K8S_CLUSTER_0_ZONE}" \ 3 --num-nodes="${K8S_CLUSTER_0_NUMBER_OF_NODES}" \ 4 --machine-type "${K8S_CLUSTER_0_MACHINE_TYPE}" \ 5 --tags=mongodb \ 6 ${GKE_SPOT_INSTANCES_SWITCH:+${GKE_SPOT_INSTANCES_SWITCH}}
1 gcloud container clusters create "${K8S_CLUSTER_1}" \ 2 --zone="${K8S_CLUSTER_1_ZONE}" \ 3 --num-nodes="${K8S_CLUSTER_1_NUMBER_OF_NODES}" \ 4 --machine-type "${K8S_CLUSTER_1_MACHINE_TYPE}" \ 5 --tags=mongodb \ 6 ${GKE_SPOT_INSTANCES_SWITCH:+${GKE_SPOT_INSTANCES_SWITCH}}
1 gcloud container clusters create "${K8S_CLUSTER_2}" \ 2 --zone="${K8S_CLUSTER_2_ZONE}" \ 3 --num-nodes="${K8S_CLUSTER_2_NUMBER_OF_NODES}" \ 4 --machine-type "${K8S_CLUSTER_2_MACHINE_TYPE}" \ 5 --tags=mongodb \ 6 ${GKE_SPOT_INSTANCES_SWITCH:+${GKE_SPOT_INSTANCES_SWITCH}}
Store credentials in your kubeconfig file.
Obtain credentials and save contexts to the current kubeconfig file. By default, this file is located in the ~/.kube/config directory and referenced by the $KUBECONFIG environment variable.
1 gcloud container clusters get-credentials "${K8S_CLUSTER_0}" --zone="${K8S_CLUSTER_0_ZONE}" 2 gcloud container clusters get-credentials "${K8S_CLUSTER_1}" --zone="${K8S_CLUSTER_1_ZONE}" 3 gcloud container clusters get-credentials "${K8S_CLUSTER_2}" --zone="${K8S_CLUSTER_2_ZONE}"
All kubectl commands reference these contexts using the following variables:
$K8S_CLUSTER_0_CONTEXT_NAME$K8S_CLUSTER_1_CONTEXT_NAME$K8S_CLUSTER_2_CONTEXT_NAME
Verify that kubectl has access to Kubernetes clusters.
1 echo "Nodes in cluster ${K8S_CLUSTER_0_CONTEXT_NAME}" 2 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" get nodes 3 echo; echo "Nodes in cluster ${K8S_CLUSTER_1_CONTEXT_NAME}" 4 kubectl --context "${K8S_CLUSTER_1_CONTEXT_NAME}" get nodes 5 echo; echo "Nodes in cluster ${K8S_CLUSTER_2_CONTEXT_NAME}" 6 kubectl --context "${K8S_CLUSTER_2_CONTEXT_NAME}" get nodes
1 Nodes in cluster gke_scratch-kubernetes-team_europe-central2-a_k8s-mdb-0-6aa01fd8df4x772f980-1668 2 NAME STATUS ROLES AGE VERSION 3 gke-k8s-mdb-0-6aa01fd8df-default-pool-54114de2-0jv5 Ready <none> 3m v1.35.7-gke.1150000 4 gke-k8s-mdb-0-6aa01fd8df-default-pool-54114de2-gjgw Ready <none> 2m55s v1.35.7-gke.1150000 5 gke-k8s-mdb-0-6aa01fd8df-default-pool-54114de2-hwjw Ready <none> 3m v1.35.7-gke.1150000 6 7 Nodes in cluster gke_scratch-kubernetes-team_europe-central2-b_k8s-mdb-1-6aa01fd8df4x772f980-1668 8 NAME STATUS ROLES AGE VERSION 9 gke-k8s-mdb-1-6aa01fd8df-default-pool-003bd080-5dtq Ready <none> 80s v1.35.7-gke.1150000 10 gke-k8s-mdb-1-6aa01fd8df-default-pool-003bd080-fz0d Ready <none> 80s v1.35.7-gke.1150000 11 gke-k8s-mdb-1-6aa01fd8df-default-pool-003bd080-qg3n Ready <none> 77s v1.35.7-gke.1150000 12 13 Nodes in cluster gke_scratch-kubernetes-team_europe-central2-c_k8s-mdb-2-6aa01fd8df4x772f980-1668 14 NAME STATUS ROLES AGE VERSION 15 gke-k8s-mdb-2-6aa01fd8df-default-pool-b6805d5a-rqjw Ready <none> 31s v1.35.7-gke.1150000