Kubernetes is not vendor specific; however, for the sake of providing explicit and complete instructions for deploying all elements of a multi-cluster Kubernetes Operator implementation, this section walks you through the process of deploying multiple Kubernetes clusters on GCP specifically.
Prerequisites
Before you begin, perform the following actions:
Create a GCP account.
Prepare a GCP project so that you can use it to create GKE (Google Kubernetes Engine) clusters.
Install the GCP CLI.
Authenticate with the
gcloud auth logincommand.Update as needed and set the environment variables defined in the following
env_variables.shfile.
1 # GCP project name - this is the only parameter that is mandatory to change here 2 export MDB_GKE_PROJECT="${MDB_GKE_PROJECT:="### Set your GKE project name here ###"}" 3 4 # Adjust the values for each Kubernetes cluster in your deployment. 5 # The deployment script references the following variables to get values for each cluster. 6 export K8S_CLUSTER_0="k8s-mdb-0${K8S_CLUSTER_SUFFIX:-""}" 7 export K8S_CLUSTER_0_ZONE="europe-central2-a" 8 export K8S_CLUSTER_0_NUMBER_OF_NODES=3 9 export K8S_CLUSTER_0_MACHINE_TYPE="e2-standard-4" 10 export K8S_CLUSTER_0_CONTEXT_NAME="gke_${MDB_GKE_PROJECT}_${K8S_CLUSTER_0_ZONE}_${K8S_CLUSTER_0}" 11 12 export K8S_CLUSTER_1="k8s-mdb-1${K8S_CLUSTER_SUFFIX:-""}" 13 export K8S_CLUSTER_1_ZONE="europe-central2-b" 14 export K8S_CLUSTER_1_NUMBER_OF_NODES=3 15 export K8S_CLUSTER_1_MACHINE_TYPE="e2-standard-4" 16 export K8S_CLUSTER_1_CONTEXT_NAME="gke_${MDB_GKE_PROJECT}_${K8S_CLUSTER_1_ZONE}_${K8S_CLUSTER_1}" 17 18 export K8S_CLUSTER_2="k8s-mdb-2${K8S_CLUSTER_SUFFIX:-""}" 19 export K8S_CLUSTER_2_ZONE="europe-central2-c" 20 export K8S_CLUSTER_2_NUMBER_OF_NODES=1 21 export K8S_CLUSTER_2_MACHINE_TYPE="e2-standard-4" 22 export K8S_CLUSTER_2_CONTEXT_NAME="gke_${MDB_GKE_PROJECT}_${K8S_CLUSTER_2_ZONE}_${K8S_CLUSTER_2}" 23 24 # Preemptible nodes disabled — caused random mid-test evictions in CI. 25 # export GKE_SPOT_INSTANCES_SWITCH="--preemptible"
Source Code
You can find all included source code in the MongoDB Kubernetes Operator repository.
Procedure
Create Kubernetes clusters.
Create three GKE (Google Kubernetes Engine) clusters. The Kubernetes Operator is deployed to one of these clusters, and MongoDB custom resources are deployed across all clusters, and are managed by the Kubernetes Operator.
1 gcloud container clusters create "${K8S_CLUSTER_0}" \ 2 --zone="${K8S_CLUSTER_0_ZONE}" \ 3 --num-nodes="${K8S_CLUSTER_0_NUMBER_OF_NODES}" \ 4 --machine-type "${K8S_CLUSTER_0_MACHINE_TYPE}" \ 5 --tags=mongodb \ 6 ${GKE_SPOT_INSTANCES_SWITCH:+${GKE_SPOT_INSTANCES_SWITCH}}
1 gcloud container clusters create "${K8S_CLUSTER_1}" \ 2 --zone="${K8S_CLUSTER_1_ZONE}" \ 3 --num-nodes="${K8S_CLUSTER_1_NUMBER_OF_NODES}" \ 4 --machine-type "${K8S_CLUSTER_1_MACHINE_TYPE}" \ 5 --tags=mongodb \ 6 ${GKE_SPOT_INSTANCES_SWITCH:+${GKE_SPOT_INSTANCES_SWITCH}}
1 gcloud container clusters create "${K8S_CLUSTER_2}" \ 2 --zone="${K8S_CLUSTER_2_ZONE}" \ 3 --num-nodes="${K8S_CLUSTER_2_NUMBER_OF_NODES}" \ 4 --machine-type "${K8S_CLUSTER_2_MACHINE_TYPE}" \ 5 --tags=mongodb \ 6 ${GKE_SPOT_INSTANCES_SWITCH:+${GKE_SPOT_INSTANCES_SWITCH}}
Store credentials in your kubeconfig file.
Obtain credentials and save contexts to the current kubeconfig file. By default, this file is located in the ~/.kube/config directory and referenced by the $KUBECONFIG environment variable.
1 gcloud container clusters get-credentials "${K8S_CLUSTER_0}" --zone="${K8S_CLUSTER_0_ZONE}" 2 gcloud container clusters get-credentials "${K8S_CLUSTER_1}" --zone="${K8S_CLUSTER_1_ZONE}" 3 gcloud container clusters get-credentials "${K8S_CLUSTER_2}" --zone="${K8S_CLUSTER_2_ZONE}"
All kubectl commands reference these contexts using the following variables:
$K8S_CLUSTER_0_CONTEXT_NAME$K8S_CLUSTER_1_CONTEXT_NAME$K8S_CLUSTER_2_CONTEXT_NAME
Verify that kubectl has access to Kubernetes clusters.
1 echo "Nodes in cluster ${K8S_CLUSTER_0_CONTEXT_NAME}" 2 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" get nodes 3 echo; echo "Nodes in cluster ${K8S_CLUSTER_1_CONTEXT_NAME}" 4 kubectl --context "${K8S_CLUSTER_1_CONTEXT_NAME}" get nodes 5 echo; echo "Nodes in cluster ${K8S_CLUSTER_2_CONTEXT_NAME}" 6 kubectl --context "${K8S_CLUSTER_2_CONTEXT_NAME}" get nodes
1 Nodes in cluster gke_scratch-kubernetes-team_europe-central2-a_k8s-mdb-0-6a6c58dd72bx7763665-6665 2 NAME STATUS ROLES AGE VERSION 3 gke-k8s-mdb-0-6a6c58dd72-default-pool-cd0366b8-296n Ready <none> 85s v1.35.6-gke.1127000 4 gke-k8s-mdb-0-6a6c58dd72-default-pool-cd0366b8-pzxm Ready <none> 73s v1.35.6-gke.1127000 5 gke-k8s-mdb-0-6a6c58dd72-default-pool-cd0366b8-q0jv Ready <none> 2m9s v1.35.6-gke.1127000 6 7 Nodes in cluster gke_scratch-kubernetes-team_europe-central2-b_k8s-mdb-1-6a6c58dd72bx7763665-6665 8 NAME STATUS ROLES AGE VERSION 9 gke-k8s-mdb-1-6a6c58dd72-default-pool-0aedc8f8-0xvr Ready <none> 46s v1.35.6-gke.1127000 10 gke-k8s-mdb-1-6a6c58dd72-default-pool-0aedc8f8-q5n7 Ready <none> 48s v1.35.6-gke.1127000 11 gke-k8s-mdb-1-6a6c58dd72-default-pool-0aedc8f8-wtgj Ready <none> 43s v1.35.6-gke.1127000 12 13 Nodes in cluster gke_scratch-kubernetes-team_europe-central2-c_k8s-mdb-2-6a6c58dd72bx7763665-6665 14 NAME STATUS ROLES AGE VERSION 15 gke-k8s-mdb-2-6a6c58dd72-default-pool-f368616d-pncw Ready <none> 30s v1.35.6-gke.1250000