O Ops Manager é responsável por facilitar volumes de trabalho, como backup de dados, monitorar o desempenho do banco de dados e muito mais. Para tornar o Ops Manager de vários clusters e o sistema de banco de dados de aplicativos resilientes a falhas de data center ou de zona inteiras, implemente o aplicativo Ops Manager e o sistema de banco de dados de aplicativos em vários clusters do Kubernetes.
Pré-requisitos
Antes de iniciar o procedimento a seguir, execute as seguintes ações:
Instalar
kubectl.Conclua o procedimento GKE Clusters ou equivalente.
Conclua o procedimento Certificados TLS ou equivalente.
Conclua o procedimento ExternoDNS ou equivalente.
Conclua o procedimento Implementar o operador MongoDB.
Configure as variáveis de ambiente necessárias da seguinte maneira:
# This script builds on top of the environment configured in the setup guides.# It depends (uses) the following env variables defined there to work correctly.# If you don't use the setup guide to bootstrap the environment, then define them here.# ${K8S_CLUSTER_0_CONTEXT_NAME}# ${K8S_CLUSTER_1_CONTEXT_NAME}# ${K8S_CLUSTER_2_CONTEXT_NAME}# ${OM_NAMESPACE}# ${CUSTOM_DOMAIN}# ${DNS_ZONE}
export S3_OPLOG_BUCKET_NAME=s3-oplog-storeexport S3_SNAPSHOT_BUCKET_NAME=s3-snapshot-store
# If you use your own S3 storage - set the values accordingly.# By default we install Minio to handle S3 storage and here are set the default credentials.export S3_ENDPOINT="minio.tenant-tiny.svc.cluster.local"export S3_ACCESS_KEY="console"export S3_SECRET_KEY="console123"
export OPS_MANAGER_VERSION="8.0.5"export APPDB_VERSION="8.0.5-ent"
export OPS_MANAGER_EXTERNAL_DOMAIN="opsmanager.${CUSTOM_DOMAIN}"export APPDB_CLUSTER_0_EXTERNAL_DOMAIN="${K8S_CLUSTER_0}.${CUSTOM_DOMAIN}"export APPDB_CLUSTER_1_EXTERNAL_DOMAIN="${K8S_CLUSTER_1}.${CUSTOM_DOMAIN}"export APPDB_CLUSTER_2_EXTERNAL_DOMAIN="${K8S_CLUSTER_2}.${CUSTOM_DOMAIN}"código fonte
Você pode encontrar todo o código-fonte incluído no repositório do MongoDB Kubernetes Operator.
Procedimento
Gerar certificados TLS.
kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" apply -f - <<EOFapiVersion: cert-manager.io/v1kind: Certificatemetadata: name: om-certspec: dnsNames: - ${OPS_MANAGER_EXTERNAL_DOMAIN} duration: 240h0m0s issuerRef: name: my-ca-issuer kind: ClusterIssuer renewBefore: 120h0m0s secretName: cert-prefix-om-cert usages: - server auth - client auth---apiVersion: cert-manager.io/v1kind: Certificatemetadata: name: om-db-certspec: dnsNames: - "*.${APPDB_CLUSTER_0_EXTERNAL_DOMAIN}" - "*.${APPDB_CLUSTER_1_EXTERNAL_DOMAIN}" - "*.${APPDB_CLUSTER_2_EXTERNAL_DOMAIN}" duration: 240h0m0s issuerRef: name: my-ca-issuer kind: ClusterIssuer renewBefore: 120h0m0s secretName: cert-prefix-om-db-cert usages: - server auth - client authEOFAdicione o certificado TLS ao GCP.
mkdir -p certs
kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get secret cert-prefix-om-cert -o jsonpath="{.data['tls\.crt']}" | base64 --decode > certs/tls.crtkubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get secret cert-prefix-om-cert -o jsonpath="{.data['tls\.key']}" | base64 --decode > certs/tls.key
gcloud compute ssl-certificates create om-certificate --certificate=certs/tls.crt --private-key=certs/tls.keyCrie os componentes necessários do Kubernetes para um balanceador de carga .
Esse balanceador de carga distribui o tráfego entre todas as réplicas do Ops Manager em todos os clusters 3.
gcloud compute firewall-rules create fw-ops-manager-hc \ --action=allow \ --direction=ingress \ --target-tags=mongodb \ --source-ranges=130.211.0.0/22,35.191.0.0/16 \ --rules=tcp:8443
gcloud compute health-checks create https om-healthcheck \ --use-serving-port \ --request-path=/monitor/health
gcloud compute backend-services create om-backend-service \ --protocol HTTPS \ --health-checks om-healthcheck \ --global
gcloud compute url-maps create om-url-map \ --default-service om-backend-service
gcloud compute target-https-proxies create om-lb-proxy \ --url-map om-url-map \ --ssl-certificates=om-certificate
gcloud compute forwarding-rules create om-forwarding-rule \ --global \ --target-https-proxy=om-lb-proxy \ --ports=443NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLEDfw-ops-manager-hc default INGRESS 1000 tcp:8443 FalseNAME PROTOCOLom-healthcheck HTTPSNAME BACKENDS PROTOCOLom-backend-service HTTPSNAME DEFAULT_SERVICEom-url-map backendServices/om-backend-serviceNAME SSL_CERTIFICATES URL_MAP REGION CERTIFICATE_MAPom-lb-proxy om-certificate om-url-mapAdicione um registro "A" à sua zona de DNS com seu domínio externo.
ip_address=$(gcloud compute forwarding-rules describe om-forwarding-rule --global --format="get(IPAddress)")
gcloud dns record-sets create "${OPS_MANAGER_EXTERNAL_DOMAIN}" --zone="${DNS_ZONE}" --type="A" --ttl="300" --rrdatas="${ip_address}"Crie credenciais para o usuário administrador do Ops Manager.
kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" --namespace "${OM_NAMESPACE}" create secret generic om-admin-user-credentials \ --from-literal=Username="admin" \ --from-literal=Password="Passw0rd@" \ --from-literal=FirstName="Jane" \ --from-literal=LastName="Doe"Implemente o Ops Manager.
kubectl apply --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" -f - <<EOFapiVersion: mongodb.com/v1kind: MongoDBOpsManagermetadata: name: omspec: topology: MultiCluster version: "${OPS_MANAGER_VERSION}" adminCredentials: om-admin-user-credentials externalConnectivity: type: ClusterIP annotations: cloud.google.com/neg: '{"exposed_ports": {"8443":{}}}' opsManagerURL: "https://${OPS_MANAGER_EXTERNAL_DOMAIN}" security: certsSecretPrefix: cert-prefix tls: ca: ca-issuer clusterSpecList: - clusterName: "${K8S_CLUSTER_0_CONTEXT_NAME}" members: 1 - clusterName: "${K8S_CLUSTER_1_CONTEXT_NAME}" members: 2 applicationDatabase: version: "${APPDB_VERSION}" topology: MultiCluster security: certsSecretPrefix: cert-prefix tls: ca: ca-issuer clusterSpecList: - clusterName: "${K8S_CLUSTER_0_CONTEXT_NAME}" members: 2 externalAccess: externalDomain: "${APPDB_CLUSTER_0_EXTERNAL_DOMAIN}" externalService: annotations: external-dns.alpha.kubernetes.io/hostname: "{podName}.${APPDB_CLUSTER_0_EXTERNAL_DOMAIN}" - clusterName: "${K8S_CLUSTER_1_CONTEXT_NAME}" members: 2 externalAccess: externalDomain: "${APPDB_CLUSTER_1_EXTERNAL_DOMAIN}" externalService: annotations: external-dns.alpha.kubernetes.io/hostname: "{podName}.${APPDB_CLUSTER_1_EXTERNAL_DOMAIN}" - clusterName: "${K8S_CLUSTER_2_CONTEXT_NAME}" members: 1 externalAccess: externalDomain: "${APPDB_CLUSTER_2_EXTERNAL_DOMAIN}" externalService: annotations: external-dns.alpha.kubernetes.io/hostname: "{podName}.${APPDB_CLUSTER_2_EXTERNAL_DOMAIN}" backup: enabled: falseEOFAguarde até que o operador Kubernetes entre em um estado pendente.
Aguarde a conclusão das implantações do banco de dados de aplicativos e do Ops Manager.
echo "Waiting for Application Database to reach Pending phase..."kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.applicationDatabase.phase}'=Pending opsmanager/om --timeout=30s
echo "Waiting for Ops Manager to reach Pending phase..."kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.opsManager.phase}'=Pending opsmanager/om --timeout=600sWaiting for Application Database to reach Pending phase...mongodbopsmanager.mongodb.com/om condition metWaiting for Ops Manager to reach Pending phase...mongodbopsmanager.mongodb.com/om condition metConfigure serviços de balanceador de carga.
svcneg0=$(kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get svcneg -o=jsonpath='{.items[0].metadata.name}')
gcloud compute backend-services add-backend om-backend-service \ --global \ --network-endpoint-group="${svcneg0}" \ --network-endpoint-group-zone="${K8S_CLUSTER_0_ZONE}" \ --balancing-mode RATE --max-rate-per-endpoint 5svcneg1=$(kubectl --context "${K8S_CLUSTER_1_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get svcneg -o=jsonpath='{.items[0].metadata.name}')
gcloud compute backend-services add-backend om-backend-service \ --global \ --network-endpoint-group="${svcneg1}" \ --network-endpoint-group-zone="${K8S_CLUSTER_1_ZONE}" \ --balancing-mode RATE --max-rate-per-endpoint 5Aguarde o Ops Manager entrar em um estado de execução.
echo "Waiting for Application Database to reach Running phase..."kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.applicationDatabase.phase}'=Running opsmanager/om --timeout=900secho; echo "Waiting for Ops Manager to reach Running phase..."kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.opsManager.phase}'=Running opsmanager/om --timeout=900secho; echo "MongoDBOpsManager resource"kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get opsmanager/omecho; echo "Pods running in cluster ${K8S_CLUSTER_0_CONTEXT_NAME}"kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get podsecho; echo "Pods running in cluster ${K8S_CLUSTER_1_CONTEXT_NAME}"kubectl --context "${K8S_CLUSTER_1_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get podsWaiting for Application Database to reach Running phase...mongodbopsmanager.mongodb.com/om condition met
Waiting for Ops Manager to reach Running phase...mongodbopsmanager.mongodb.com/om condition met
MongoDBOpsManager resourceNAME REPLICAS VERSION STATE (OPSMANAGER) STATE (APPDB) STATE (BACKUP) AGE WARNINGSom 8.0.5 Running Running Disabled 14m
Pods running in cluster gke_scratch-kubernetes-team_europe-central2-a_k8s-mdb-0-68c29e4f7014ee0007f0695a-5538NAME READY STATUS RESTARTS AGEom-0-0 1/1 Running 0 11mom-db-0-0 3/3 Running 0 4m4som-db-0-1 3/3 Running 0 5m11s
Pods running in cluster gke_scratch-kubernetes-team_europe-central2-b_k8s-mdb-1-68c29e4f7014ee0007f0695a-5538NAME READY STATUS RESTARTS AGEom-1-0 1/1 Running 0 11mom-1-1 1/1 Running 0 8m30som-db-1-0 3/3 Running 0 117som-db-1-1 3/3 Running 0 3m5sInstale o Minio.
kubectl kustomize "github.com/minio/operator/resources/?timeout=120&ref=v5.0.12" | \ kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" apply -f -
kubectl kustomize "github.com/minio/operator/examples/kustomization/tenant-tiny?timeout=120&ref=v5.0.12" | \ kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" apply -f -
# add two buckets to the tenant configkubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "tenant-tiny" patch tenant/myminio \ --type='json' \ -p="[{\"op\": \"add\", \"path\": \"/spec/buckets\", \"value\": [{\"name\": \"${S3_OPLOG_BUCKET_NAME}\"}, {\"name\": \"${S3_SNAPSHOT_BUCKET_NAME}\"}]}]"Configure o Kubernetes Secrets para backups do Ops Manager.
kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" create secret generic s3-access-secret \ --from-literal=accessKey="${S3_ACCESS_KEY}" \ --from-literal=secretKey="${S3_SECRET_KEY}"
# minio TLS secrets are signed with the default k8s root CAkubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" create secret generic s3-ca-cert \ --from-literal=ca.crt="$(kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n kube-system get configmap kube-root-ca.crt -o jsonpath="{.data.ca\.crt}")"Ative os backups S3 (Minio) no Ops Manager.
kubectl apply --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" -f - <<EOFapiVersion: mongodb.com/v1kind: MongoDBOpsManagermetadata: name: omspec: topology: MultiCluster version: "${OPS_MANAGER_VERSION}" adminCredentials: om-admin-user-credentials externalConnectivity: type: ClusterIP annotations: cloud.google.com/neg: '{"exposed_ports": {"8443":{}}}' opsManagerURL: "https://${OPS_MANAGER_EXTERNAL_DOMAIN}" security: certsSecretPrefix: cert-prefix tls: ca: ca-issuer clusterSpecList: - clusterName: "${K8S_CLUSTER_0_CONTEXT_NAME}" members: 1 backup: members: 0 - clusterName: "${K8S_CLUSTER_1_CONTEXT_NAME}" members: 2 backup: members: 0 - clusterName: "${K8S_CLUSTER_2_CONTEXT_NAME}" members: 0 backup: members: 1 applicationDatabase: version: "${APPDB_VERSION}" topology: MultiCluster security: certsSecretPrefix: cert-prefix tls: ca: ca-issuer clusterSpecList: - clusterName: "${K8S_CLUSTER_0_CONTEXT_NAME}" members: 2 externalAccess: externalDomain: "${APPDB_CLUSTER_0_EXTERNAL_DOMAIN}" externalService: annotations: external-dns.alpha.kubernetes.io/hostname: "{podName}.${APPDB_CLUSTER_0_EXTERNAL_DOMAIN}" - clusterName: "${K8S_CLUSTER_1_CONTEXT_NAME}" members: 2 externalAccess: externalDomain: "${APPDB_CLUSTER_1_EXTERNAL_DOMAIN}" externalService: annotations: external-dns.alpha.kubernetes.io/hostname: "{podName}.${APPDB_CLUSTER_1_EXTERNAL_DOMAIN}" - clusterName: "${K8S_CLUSTER_2_CONTEXT_NAME}" members: 1 externalAccess: externalDomain: "${APPDB_CLUSTER_2_EXTERNAL_DOMAIN}" externalService: annotations: external-dns.alpha.kubernetes.io/hostname: "{podName}.${APPDB_CLUSTER_2_EXTERNAL_DOMAIN}" backup: enabled: true s3Stores: - name: my-s3-block-store s3SecretRef: name: "s3-access-secret" pathStyleAccessEnabled: true s3BucketEndpoint: "${S3_ENDPOINT}" s3BucketName: "${S3_SNAPSHOT_BUCKET_NAME}" customCertificateSecretRefs: - name: s3-ca-cert key: ca.crt s3OpLogStores: - name: my-s3-oplog-store s3SecretRef: name: "s3-access-secret" s3BucketEndpoint: "${S3_ENDPOINT}" s3BucketName: "${S3_OPLOG_BUCKET_NAME}" pathStyleAccessEnabled: true customCertificateSecretRefs: - name: s3-ca-cert key: ca.crtEOFAguarde o Ops Manager entrar em um estado de execução.
echo; echo "Waiting for Backup to reach Running phase..."kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.backup.phase}'=Running opsmanager/om --timeout=1200secho "Waiting for Application Database to reach Running phase..."kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.applicationDatabase.phase}'=Running opsmanager/om --timeout=600secho; echo "Waiting for Ops Manager to reach Running phase..."kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.opsManager.phase}'=Running opsmanager/om --timeout=600secho; echo "MongoDBOpsManager resource"kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get opsmanager/omecho; echo "Pods running in cluster ${K8S_CLUSTER_0_CONTEXT_NAME}"kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get podsecho; echo "Pods running in cluster ${K8S_CLUSTER_1_CONTEXT_NAME}"kubectl --context "${K8S_CLUSTER_1_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get podsecho; echo "Pods running in cluster ${K8S_CLUSTER_2_CONTEXT_NAME}"kubectl --context "${K8S_CLUSTER_2_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get podsWaiting for Backup to reach Running phase...mongodbopsmanager.mongodb.com/om condition metWaiting for Application Database to reach Running phase...mongodbopsmanager.mongodb.com/om condition met
Waiting for Ops Manager to reach Running phase...mongodbopsmanager.mongodb.com/om condition met
MongoDBOpsManager resourceNAME REPLICAS VERSION STATE (OPSMANAGER) STATE (APPDB) STATE (BACKUP) AGE WARNINGSom 8.0.5 Running Running Running 18m
Pods running in cluster gke_scratch-kubernetes-team_europe-central2-a_k8s-mdb-0-68c29e4f7014ee0007f0695a-5538NAME READY STATUS RESTARTS AGEom-0-0 1/1 Running 0 15mom-db-0-0 3/3 Running 0 8m7som-db-0-1 3/3 Running 0 9m14s
Pods running in cluster gke_scratch-kubernetes-team_europe-central2-b_k8s-mdb-1-68c29e4f7014ee0007f0695a-5538NAME READY STATUS RESTARTS AGEom-1-0 1/1 Running 0 15mom-1-1 1/1 Running 0 12mom-db-1-0 3/3 Running 0 6m1som-db-1-1 3/3 Running 0 7m9s
Pods running in cluster gke_scratch-kubernetes-team_europe-central2-c_k8s-mdb-2-68c29e4f7014ee0007f0695a-5538NAME READY STATUS RESTARTS AGEom-2-backup-daemon-0 1/1 Running 0 3m16som-db-2-0 3/3 Running 0 5mCrie a organização MongoDB e obtenha credenciais.
Para configurar as credenciais, você deve criar uma organização do Ops Manager, gerar chaves de API programáticas na interface do usuário do Ops Manager e criar um segredo com o IP do Load Balancer. Consulte Criar Credenciais para o Operador Kubernetes para saber mais.