MongoDB Ops Manager负责促进备份数据、监控数据库性能等工作负载。 为了使多集群MongoDB Ops Manager和应用程序数据库部署能够应对整个数据中心或区域故障，请在多个Kubernetes集群上部署MongoDB Ops Manager应用程序和应用程序数据库。
先决条件
在开始以下过程之前，请执行以下操作：
安装
kubectl。完成 GKE 集群过程或类似步骤。
完成 TLS 证书程序 或同等程序。
完成 Istio 服务网格程序 或同等程序。
完成部署MongoDB Operator程序。
按如下方式设置所需的环境变量：
# This script builds on top of the environment configured in the setup guides. # It depends (uses) the following env variables defined there to work correctly. # If you don't use the setup guide to bootstrap the environment, then define them here. # ${K8S_CLUSTER_0_CONTEXT_NAME} # ${K8S_CLUSTER_1_CONTEXT_NAME} # ${K8S_CLUSTER_2_CONTEXT_NAME} # ${OM_NAMESPACE} export S3_OPLOG_BUCKET_NAME=s3-oplog-store export S3_SNAPSHOT_BUCKET_NAME=s3-snapshot-store # If you use your own S3 storage - set the values accordingly. # By default we install Minio to handle S3 storage and here are set the default credentials. export S3_ENDPOINT="minio.tenant-tiny.svc.cluster.local" export S3_ACCESS_KEY="console" export S3_SECRET_KEY="console123" export OPS_MANAGER_VERSION="8.0.5" export APPDB_VERSION="8.0.5-ent"
源代码
您可以在 MongoDB Kubernetes Operator存储库中找到所有包含的源代码。
步骤
1
生成 TLS 证书。
kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" apply -f - <<EOF apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: om-cert spec: dnsNames: - om-svc.${OM_NAMESPACE}.svc.cluster.local duration: 240h0m0s issuerRef: name: my-ca-issuer kind: ClusterIssuer renewBefore: 120h0m0s secretName: cert-prefix-om-cert usages: - server auth - client auth --- apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: om-db-cert spec: dnsNames: - "*.${OM_NAMESPACE}.svc.cluster.local" duration: 240h0m0s issuerRef: name: my-ca-issuer kind: ClusterIssuer renewBefore: 120h0m0s secretName: cert-prefix-om-db-cert usages: - server auth - client auth EOF
2
安装 Ops Manager
此时，您已准备好用于部署 资源的环境和Kubernetes MongoDB Ops ManagerOperator。
为MongoDB Ops Manager管理员用户创建必要的档案， Kubernetes Operator 在部署MongoDB Ops Manager应用程序实例后将创建该用户：
1 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" --namespace "${OM_NAMESPACE}" create secret generic om-admin-user-credentials \ 2 --from-literal=Username="admin" \ 3 --from-literal=Password="Passw0rd@" \ 4 --from-literal=FirstName="Jane" \ 5 --from-literal=LastName="Doe" 在单个成员集群（也称为操作符集群）上部署尽可能简单的
MongoDBOpsManager自定义资源（启用 TLS）。此部署与单集群模式的部署几乎相同，但将
spec.topology和spec.applicationDatabase.topology设立为MultiCluster。这种部署方式表明，单个 Kubernetes 集群部署是单个 Kubernetes 成员集群上的多 Kubernetes 集群部署的特例。 MongoDB Ops ManagerKubernetes您可以从一开始就在所需数量的 集群上部署 应用程序和应用程序数据库，而不必从只有单个成员Kubernetes 集群的部署开始。
此时，您已准备好MongoDB Ops Manager部署以跨越多个Kubernetes集群，本过程稍后将执行此操作。
1 kubectl apply --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" -f - <<EOF 2 apiVersion: mongodb.com/v1 3 kind: MongoDBOpsManager 4 metadata: 5 name: om 6 spec: 7 topology: MultiCluster 8 version: "${OPS_MANAGER_VERSION}" 9 adminCredentials: om-admin-user-credentials 10 externalConnectivity: 11 type: LoadBalancer 12 security: 13 certsSecretPrefix: cert-prefix 14 tls: 15 ca: ca-issuer 16 clusterSpecList: 17 - clusterName: "${K8S_CLUSTER_0_CONTEXT_NAME}" 18 members: 1 19 applicationDatabase: 20 version: "${APPDB_VERSION}" 21 topology: MultiCluster 22 security: 23 certsSecretPrefix: cert-prefix 24 tls: 25 ca: ca-issuer 26 clusterSpecList: 27 - clusterName: "${K8S_CLUSTER_0_CONTEXT_NAME}" 28 members: 3 29 backup: 30 enabled: false 31 EOF 等待 Kubernetes Operator 接手工作并进入
status.applicationDatabase.phase=Pending状态。 等待应用程序数据库和MongoDB Ops Manager部署完成。1 echo "Waiting for Application Database to reach Pending phase..." 2 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.applicationDatabase.phase}'=Pending opsmanager/om --timeout=30s 1 Waiting for Application Database to reach Pending phase... 2 mongodbopsmanager.mongodb.com/om condition met 部署MongoDB Ops Manager 。 Kubernetes Operator 通过执行以下步骤来部署MongoDB Ops Manager 。 它：
部署应用程序数据库的副本集节点，并等待副本集中的 MongoDB 进程开始运行。
使用应用程序数据库的连接 部署MongoDB Ops Managerstring 应用程序实例，并等待其准备就绪。
将监控MongoDB Agent容器添加到每个应用程序数据库的 Pod。
等待MongoDB Ops Manager应用程序和应用程序数据库 Pod 开始运行。
1 echo "Waiting for Application Database to reach Running phase..." 2 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.applicationDatabase.phase}'=Running opsmanager/om --timeout=900s 3 echo; echo "Waiting for Ops Manager to reach Running phase..." 4 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.opsManager.phase}'=Running opsmanager/om --timeout=900s 5 echo; echo "MongoDBOpsManager resource" 6 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get opsmanager/om 7 echo; echo "Pods running in cluster ${K8S_CLUSTER_0_CONTEXT_NAME}" 8 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get pods 9 echo; echo "Pods running in cluster ${K8S_CLUSTER_1_CONTEXT_NAME}" 10 kubectl --context "${K8S_CLUSTER_1_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get pods 1 Waiting for Application Database to reach Running phase... 2 mongodbopsmanager.mongodb.com/om condition met 3 4 Waiting for Ops Manager to reach Running phase... 5 mongodbopsmanager.mongodb.com/om condition met 6 7 MongoDBOpsManager resource 8 NAME REPLICAS VERSION STATE (OPSMANAGER) STATE (APPDB) STATE (BACKUP) AGE WARNINGS 9 om 8.0.5 Running Running Disabled 12m 10 11 Pods running in cluster gke_scratch-kubernetes-team_europe-central2-a_k8s-mdb-0-682f2df6e1745e000788a1d5-24552 12 NAME READY STATUS RESTARTS AGE 13 om-0-0 2/2 Running 0 9m41s 14 om-db-0-0 4/4 Running 0 51s 15 om-db-0-1 4/4 Running 0 2m25s 16 om-db-0-2 4/4 Running 0 4m16s 17 18 Pods running in cluster gke_scratch-kubernetes-team_europe-central2-b_k8s-mdb-1-682f2df6e1745e000788a1d5-24552 现在您已经以多集群模式部署了一个单成员集群，您可以重新配置此部署以跨越多个 Kubernetes 集群。
在第二个成员集群上，再部署两个应用程序数据库副本集成员和一个MongoDB Ops Manager应用程序实例：
1 kubectl apply --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" -f - <<EOF 2 apiVersion: mongodb.com/v1 3 kind: MongoDBOpsManager 4 metadata: 5 name: om 6 spec: 7 topology: MultiCluster 8 version: "${OPS_MANAGER_VERSION}" 9 adminCredentials: om-admin-user-credentials 10 externalConnectivity: 11 type: LoadBalancer 12 security: 13 certsSecretPrefix: cert-prefix 14 tls: 15 ca: ca-issuer 16 clusterSpecList: 17 - clusterName: "${K8S_CLUSTER_0_CONTEXT_NAME}" 18 members: 1 19 - clusterName: "${K8S_CLUSTER_1_CONTEXT_NAME}" 20 members: 1 21 applicationDatabase: 22 version: "${APPDB_VERSION}" 23 topology: MultiCluster 24 security: 25 certsSecretPrefix: cert-prefix 26 tls: 27 ca: ca-issuer 28 clusterSpecList: 29 - clusterName: "${K8S_CLUSTER_0_CONTEXT_NAME}" 30 members: 3 31 - clusterName: "${K8S_CLUSTER_1_CONTEXT_NAME}" 32 members: 2 33 backup: 34 enabled: false 35 EOF 等待 Kubernetes Operator 接手工作（待处理阶段）：
1 echo "Waiting for Application Database to reach Pending phase..." 2 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.applicationDatabase.phase}'=Pending opsmanager/om --timeout=30s 3 4 echo "Waiting for Ops Manager to reach Pending phase..." 5 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.opsManager.phase}'=Pending opsmanager/om --timeout=600s 1 Waiting for Application Database to reach Pending phase... 2 mongodbopsmanager.mongodb.com/om condition met 3 Waiting for Ops Manager to reach Pending phase... 4 mongodbopsmanager.mongodb.com/om condition met 等待 Kubernetes Operator 完成所有组件的部署：
1 echo "Waiting for Application Database to reach Running phase..." 2 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.applicationDatabase.phase}'=Running opsmanager/om --timeout=600s 3 echo; echo "Waiting for Ops Manager to reach Running phase..." 4 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.opsManager.phase}'=Running opsmanager/om --timeout=600s 5 echo; echo "MongoDBOpsManager resource" 6 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get opsmanager/om 7 echo; echo "Pods running in cluster ${K8S_CLUSTER_0_CONTEXT_NAME}" 8 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get pods 9 echo; echo "Pods running in cluster ${K8S_CLUSTER_1_CONTEXT_NAME}" 10 kubectl --context "${K8S_CLUSTER_1_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get pods 1 Waiting for Application Database to reach Running phase... 2 mongodbopsmanager.mongodb.com/om condition met 3 4 Waiting for Ops Manager to reach Running phase... 5 mongodbopsmanager.mongodb.com/om condition met 6 7 MongoDBOpsManager resource 8 NAME REPLICAS VERSION STATE (OPSMANAGER) STATE (APPDB) STATE (BACKUP) AGE WARNINGS 9 om 8.0.5 Running Running Disabled 20m 10 11 Pods running in cluster gke_scratch-kubernetes-team_europe-central2-a_k8s-mdb-0-682f2df6e1745e000788a1d5-24552 12 NAME READY STATUS RESTARTS AGE 13 om-0-0 2/2 Running 0 2m53s 14 om-db-0-0 4/4 Running 0 8m42s 15 om-db-0-1 4/4 Running 0 10m 16 om-db-0-2 4/4 Running 0 12m 17 18 Pods running in cluster gke_scratch-kubernetes-team_europe-central2-b_k8s-mdb-1-682f2df6e1745e000788a1d5-24552 19 NAME READY STATUS RESTARTS AGE 20 om-1-0 2/2 Running 0 3m24s 21 om-db-1-0 4/4 Running 0 7m43s 22 om-db-1-1 4/4 Running 0 5m31s
3
启用备份。
在 MongoDB Ops Manager 应用程序的多 Kubernetes 集群部署中，您只能配置基于 S3 的备份存储。此过程引用 env_variables.sh 中定义的 S3_*。
可选。 安装 MinIO Operator 。
此过程使用 MinIO Operator 为备份部署 S3 兼容存储。如果您有可用的Amazon Web Services S3 或其他 S3 兼容存储桶，则可以跳过此步骤。在这种情况下，请在 env_variables.sh 中相应地调整
S3_*变量。1 kubectl kustomize "github.com/minio/operator/resources/?timeout=120&ref=v5.0.12" | \ 2 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" apply -f - 3 4 kubectl kustomize "github.com/minio/operator/examples/kustomization/tenant-tiny?timeout=120&ref=v5.0.12" | \ 5 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" apply -f - 6 7 # add two buckets to the tenant config 8 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "tenant-tiny" patch tenant/myminio \ 9 --type='json' \ 10 -p="[{\"op\": \"add\", \"path\": \"/spec/buckets\", \"value\": [{\"name\": \"${S3_OPLOG_BUCKET_NAME}\"}, {\"name\": \"${S3_SNAPSHOT_BUCKET_NAME}\"}]}]" 11 12 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" label namespace "tenant-tiny" istio-injection=enabled --overwrite 在配置和启用备份之前，请创建密钥：
s3-access-secret— 包含S 3档案。s3-ca-cert- 包含颁发存储桶服务器证书的CA证书。 在此过程中使用的示例 MinIO 部署中，将使用默认的 Kubernetes 根CA证书对证书进行签名。 由于它不是公众信任的CA证书，因此您必须提供该证书， MongoDB Ops Manager才能信任该连接。
如果您使用公开信任的证书，则可以跳过此步骤并删除
spec.backup.s3Stores.customCertificateSecretRefs和spec.backup.s3OpLogStores.customCertificateSecretRefs设置中的值。1 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" create secret generic s3-access-secret \ 2 --from-literal=accessKey="${S3_ACCESS_KEY}" \ 3 --from-literal=secretKey="${S3_SECRET_KEY}" 4 5 # minio TLS secrets are signed with the default k8s root CA 6 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" create secret generic s3-ca-cert \ 7 --from-literal=ca.crt="$(kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n kube-system get configmap kube-root-ca.crt -o jsonpath="{.data.ca\.crt}")"
4
在启用备份的情况下重新部署MongoDB Ops Manager 。
KubernetesOperatorMongoDB Ops Manager 可以在为其配置Kubernetes Operator 的任何成员集群上以任意组合配置和部署所有组件、 应用程序、备份守护程序实例和应用程序数据库的副本集节点。
为了说明多 Kubernetes 集群部署配置的灵活性，在第三个成员集群上仅部署一个备份守护进程实例，并为第一个和第二个集群指定零个备份守护进程成员。
1 kubectl apply --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" -f - <<EOF 2 apiVersion: mongodb.com/v1 3 kind: MongoDBOpsManager 4 metadata: 5 name: om 6 spec: 7 topology: MultiCluster 8 version: "${OPS_MANAGER_VERSION}" 9 adminCredentials: om-admin-user-credentials 10 externalConnectivity: 11 type: LoadBalancer 12 security: 13 certsSecretPrefix: cert-prefix 14 tls: 15 ca: ca-issuer 16 clusterSpecList: 17 - clusterName: "${K8S_CLUSTER_0_CONTEXT_NAME}" 18 members: 1 19 backup: 20 members: 0 21 - clusterName: "${K8S_CLUSTER_1_CONTEXT_NAME}" 22 members: 1 23 backup: 24 members: 0 25 - clusterName: "${K8S_CLUSTER_2_CONTEXT_NAME}" 26 members: 0 27 backup: 28 members: 1 29 applicationDatabase: 30 version: "${APPDB_VERSION}" 31 topology: MultiCluster 32 security: 33 certsSecretPrefix: cert-prefix 34 tls: 35 ca: ca-issuer 36 clusterSpecList: 37 - clusterName: "${K8S_CLUSTER_0_CONTEXT_NAME}" 38 members: 3 39 - clusterName: "${K8S_CLUSTER_1_CONTEXT_NAME}" 40 members: 2 41 backup: 42 enabled: true 43 s3Stores: 44 - name: my-s3-block-store 45 s3SecretRef: 46 name: "s3-access-secret" 47 pathStyleAccessEnabled: true 48 s3BucketEndpoint: "${S3_ENDPOINT}" 49 s3BucketName: "${S3_SNAPSHOT_BUCKET_NAME}" 50 customCertificateSecretRefs: 51 - name: s3-ca-cert 52 key: ca.crt 53 s3OpLogStores: 54 - name: my-s3-oplog-store 55 s3SecretRef: 56 name: "s3-access-secret" 57 s3BucketEndpoint: "${S3_ENDPOINT}" 58 s3BucketName: "${S3_OPLOG_BUCKET_NAME}" 59 pathStyleAccessEnabled: true 60 customCertificateSecretRefs: 61 - name: s3-ca-cert 62 key: ca.crt 63 EOF 等待 Kubernetes Operator 完成配置：
1 echo; echo "Waiting for Backup to reach Running phase..." 2 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.backup.phase}'=Running opsmanager/om --timeout=1200s 3 echo "Waiting for Application Database to reach Running phase..." 4 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.applicationDatabase.phase}'=Running opsmanager/om --timeout=600s 5 echo; echo "Waiting for Ops Manager to reach Running phase..." 6 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.opsManager.phase}'=Running opsmanager/om --timeout=600s 7 echo; echo "MongoDBOpsManager resource" 8 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get opsmanager/om 9 echo; echo "Pods running in cluster ${K8S_CLUSTER_0_CONTEXT_NAME}" 10 kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get pods 11 echo; echo "Pods running in cluster ${K8S_CLUSTER_1_CONTEXT_NAME}" 12 kubectl --context "${K8S_CLUSTER_1_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get pods 13 echo; echo "Pods running in cluster ${K8S_CLUSTER_2_CONTEXT_NAME}" 14 kubectl --context "${K8S_CLUSTER_2_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get pods 1 Waiting for Backup to reach Running phase... 2 mongodbopsmanager.mongodb.com/om condition met 3 Waiting for Application Database to reach Running phase... 4 mongodbopsmanager.mongodb.com/om condition met 5 6 Waiting for Ops Manager to reach Running phase... 7 mongodbopsmanager.mongodb.com/om condition met 8 9 MongoDBOpsManager resource 10 NAME REPLICAS VERSION STATE (OPSMANAGER) STATE (APPDB) STATE (BACKUP) AGE WARNINGS 11 om 8.0.5 Running Running Running 23m 12 13 Pods running in cluster gke_scratch-kubernetes-team_europe-central2-a_k8s-mdb-0-682f2df6e1745e000788a1d5-24552 14 NAME READY STATUS RESTARTS AGE 15 om-0-0 2/2 Running 0 5m46s 16 om-db-0-0 4/4 Running 0 11m 17 om-db-0-1 4/4 Running 0 13m 18 om-db-0-2 4/4 Running 0 15m 19 20 Pods running in cluster gke_scratch-kubernetes-team_europe-central2-b_k8s-mdb-1-682f2df6e1745e000788a1d5-24552 21 NAME READY STATUS RESTARTS AGE 22 om-1-0 2/2 Running 0 6m17s 23 om-db-1-0 4/4 Running 0 10m 24 om-db-1-1 4/4 Running 0 8m24s 25 26 Pods running in cluster gke_scratch-kubernetes-team_europe-central2-c_k8s-mdb-2-682f2df6e1745e000788a1d5-24552 27 NAME READY STATUS RESTARTS AGE 28 om-2-backup-daemon-0 2/2 Running 0 2m31s
5
为 Kubernetes 操作符 创建档案。
要配置凭证，您必须创建一个MongoDB Ops Manager组织，在MongoDB Ops Manager用户界面中生成编程API密钥，并使用您的负载均衡器IP创建密钥。请参阅为Kubernetes Operator 创建档案以学习；了解更多信息。