El Ops Manager es responsable de facilitar cargas de trabajo como la copia de seguridad de datos, la supervisión del rendimiento de la base de datos y más. Para hacer que tu Ops Manager de múltiples clústeres y la implementación de la base de datos de la aplicación sean resilientes frente a fallos de centros de datos o zonas enteras, implementa Ops Manager Application y la base de datos de la aplicación en varios clústeres de Kubernetes.
Requisitos previos
Antes de comenzar el siguiente procedimiento, tome las siguientes acciones:
Instale
kubectl.Complete el procedimiento de clústeres de GKE o el equivalente.
Completar el procedimiento de certificados TLS o el equivalente.
Completa el procedimiento de ExternalDNS o su equivalente.
Completa el procedimiento Implementar el operador de MongoDB.
Configura las variables de entorno requeridas de la siguiente manera:
# This script builds on top of the environment configured in the setup guides. # It depends (uses) the following env variables defined there to work correctly. # If you don't use the setup guide to bootstrap the environment, then define them here. # ${K8S_CLUSTER_0_CONTEXT_NAME} # ${K8S_CLUSTER_1_CONTEXT_NAME} # ${K8S_CLUSTER_2_CONTEXT_NAME} # ${OM_NAMESPACE} # ${CUSTOM_DOMAIN} # ${DNS_ZONE} export S3_OPLOG_BUCKET_NAME=s3-oplog-store export S3_SNAPSHOT_BUCKET_NAME=s3-snapshot-store # If you use your own S3 storage - set the values accordingly. # By default we install Minio to handle S3 storage and here are set the default credentials. export S3_ENDPOINT="minio.tenant-tiny.svc.cluster.local" export S3_ACCESS_KEY="console" export S3_SECRET_KEY="console123" export OPS_MANAGER_VERSION="8.0.5" export APPDB_VERSION="8.0.5-ent" export OPS_MANAGER_EXTERNAL_DOMAIN="opsmanager.${CUSTOM_DOMAIN}" export APPDB_CLUSTER_0_EXTERNAL_DOMAIN="${K8S_CLUSTER_0}.${CUSTOM_DOMAIN}" export APPDB_CLUSTER_1_EXTERNAL_DOMAIN="${K8S_CLUSTER_1}.${CUSTOM_DOMAIN}" export APPDB_CLUSTER_2_EXTERNAL_DOMAIN="${K8S_CLUSTER_2}.${CUSTOM_DOMAIN}" # Run-scoped names for project-global GCP load balancer resources (KUBE-268). # Multiple CI runs share the same GCP project; with fixed global names, one run's # pre-clean/teardown deletes a concurrent run's load balancer mid-flight. # Derive the run-specific suffix from ${DNS_ZONE} (set in the ExternalDNS setup # guide). For docs users there is no run suffix and the names stay unchanged. lb_suffix="${DNS_ZONE#"mongodb"}" export OM_FIREWALL_RULE_NAME="fw-ops-manager-hc${lb_suffix}" export OM_HEALTHCHECK_NAME="om-healthcheck${lb_suffix}" export OM_BACKEND_SERVICE_NAME="om-backend-service${lb_suffix}" export OM_URL_MAP_NAME="om-url-map${lb_suffix}" export OM_LB_PROXY_NAME="om-lb-proxy${lb_suffix}" export OM_CERTIFICATE_NAME="om-certificate${lb_suffix}" export OM_FORWARDING_RULE_NAME="om-forwarding-rule${lb_suffix}" # Retry wrapper for gcloud commands to handle transient GCP API errors # (ConnectionError, RemoteDisconnected, etc.). Functions are inherited by # subshells, so this is available inside snippet functions run by sample_test_runner. gcloud_retry() { for attempt in 1 2 3; do if gcloud "$@"; then return 0; fi if (( attempt < 3 )); then echo "gcloud failed (attempt ${attempt}/3), retrying in $((attempt * 5))s..." >&2 sleep $((attempt * 5)) fi done return 1 }
Código fuente
Se puede encontrar todo el código fuente incluido en el repositorio de MongoDB Kubernetes operador.
Procedimiento
1
Generar certificados TLS.
kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" apply -f - <<EOF apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: om-cert spec: dnsNames: - ${OPS_MANAGER_EXTERNAL_DOMAIN} duration: 240h0m0s issuerRef: name: my-ca-issuer kind: ClusterIssuer renewBefore: 120h0m0s secretName: cert-prefix-om-cert usages: - server auth - client auth --- apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: om-db-cert spec: dnsNames: - "*.${APPDB_CLUSTER_0_EXTERNAL_DOMAIN}" - "*.${APPDB_CLUSTER_1_EXTERNAL_DOMAIN}" - "*.${APPDB_CLUSTER_2_EXTERNAL_DOMAIN}" duration: 240h0m0s issuerRef: name: my-ca-issuer kind: ClusterIssuer renewBefore: 120h0m0s secretName: cert-prefix-om-db-cert usages: - server auth - client auth EOF
2
Agrega un certificado TLS a GCP.
mkdir -p certs kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get secret cert-prefix-om-cert -o jsonpath="{.data['tls\.crt']}" | base64 --decode > certs/tls.crt kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get secret cert-prefix-om-cert -o jsonpath="{.data['tls\.key']}" | base64 --decode > certs/tls.key gcloud_retry compute ssl-certificates create "${OM_CERTIFICATE_NAME}" --certificate=certs/tls.crt --private-key=certs/tls.key
3
Crea los componentes necesarios de Kubernetes para un balanceador de carga.
Este balanceador de carga distribuye el tráfico entre todas las réplicas de Ops Manager en todos los clústeres 3.
gcloud_retry compute firewall-rules create "${OM_FIREWALL_RULE_NAME}" \ --action=allow \ --direction=ingress \ --target-tags=mongodb \ --source-ranges=130.211.0.0/22,35.191.0.0/16 \ --rules=tcp:8443 gcloud_retry compute health-checks create https "${OM_HEALTHCHECK_NAME}" \ --use-serving-port \ --request-path=/monitor/health gcloud_retry compute backend-services create "${OM_BACKEND_SERVICE_NAME}" \ --protocol HTTPS \ --health-checks "${OM_HEALTHCHECK_NAME}" \ --global gcloud_retry compute url-maps create "${OM_URL_MAP_NAME}" \ --default-service "${OM_BACKEND_SERVICE_NAME}" gcloud_retry compute target-https-proxies create "${OM_LB_PROXY_NAME}" \ --url-map "${OM_URL_MAP_NAME}" \ --ssl-certificates="${OM_CERTIFICATE_NAME}" gcloud_retry compute forwarding-rules create "${OM_FORWARDING_RULE_NAME}" \ --global \ --target-https-proxy="${OM_LB_PROXY_NAME}" \ --ports=443
NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED fw-ops-manager-hc-6aa01fd8x980-18899 default INGRESS 1000 tcp:8443 False NAME PROTOCOL om-healthcheck-6aa01fd8x980-18899 HTTPS NAME BACKENDS PROTOCOL om-backend-service-6aa01fd8x980-18899 HTTPS NAME DEFAULT_SERVICE om-url-map-6aa01fd8x980-18899 backendServices/om-backend-service-6aa01fd8x980-18899 NAME SSL_CERTIFICATES URL_MAP REGION CERTIFICATE_MAP om-lb-proxy-6aa01fd8x980-18899 om-certificate-6aa01fd8x980-18899 om-url-map-6aa01fd8x980-18899
4
Agrega un registro "A" a tu zona DNS con tu dominio externo.
ip_address=$(gcloud_retry compute forwarding-rules describe "${OM_FORWARDING_RULE_NAME}" --global --format="get(IPAddress)") gcloud_retry dns record-sets create "${OPS_MANAGER_EXTERNAL_DOMAIN}" --zone="${DNS_ZONE}" --type="A" --ttl="300" --rrdatas="${ip_address}"
5
Cree credenciales para el usuario administrador de Ops Manager.
kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" --namespace "${OM_NAMESPACE}" create secret generic om-admin-user-credentials \ --from-literal=Username="admin" \ --from-literal=Password="Passw0rd@" \ --from-literal=FirstName="Jane" \ --from-literal=LastName="Doe"
6
Implementar Ops Manager.
kubectl apply --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" -f - <<EOF apiVersion: mongodb.com/v1 kind: MongoDBOpsManager metadata: name: om spec: topology: MultiCluster version: "${OPS_MANAGER_VERSION}" adminCredentials: om-admin-user-credentials externalConnectivity: type: ClusterIP annotations: cloud.google.com/neg: '{"exposed_ports": {"8443":{}}}' opsManagerURL: "https://${OPS_MANAGER_EXTERNAL_DOMAIN}" security: certsSecretPrefix: cert-prefix tls: ca: ca-issuer clusterSpecList: - clusterName: "${K8S_CLUSTER_0_CONTEXT_NAME}" members: 1 - clusterName: "${K8S_CLUSTER_1_CONTEXT_NAME}" members: 2 applicationDatabase: version: "${APPDB_VERSION}" topology: MultiCluster security: certsSecretPrefix: cert-prefix tls: ca: ca-issuer clusterSpecList: - clusterName: "${K8S_CLUSTER_0_CONTEXT_NAME}" members: 2 externalAccess: externalDomain: "${APPDB_CLUSTER_0_EXTERNAL_DOMAIN}" externalService: annotations: external-dns.alpha.kubernetes.io/hostname: "{podName}.${APPDB_CLUSTER_0_EXTERNAL_DOMAIN}" - clusterName: "${K8S_CLUSTER_1_CONTEXT_NAME}" members: 2 externalAccess: externalDomain: "${APPDB_CLUSTER_1_EXTERNAL_DOMAIN}" externalService: annotations: external-dns.alpha.kubernetes.io/hostname: "{podName}.${APPDB_CLUSTER_1_EXTERNAL_DOMAIN}" - clusterName: "${K8S_CLUSTER_2_CONTEXT_NAME}" members: 1 externalAccess: externalDomain: "${APPDB_CLUSTER_2_EXTERNAL_DOMAIN}" externalService: annotations: external-dns.alpha.kubernetes.io/hostname: "{podName}.${APPDB_CLUSTER_2_EXTERNAL_DOMAIN}" backup: enabled: false EOF
7
Espere a que el operador de Kubernetes entre en estado pendiente.
Espere a que se completen las implementaciones de la base de datos de la aplicación y de Ops Manager.
echo "Waiting for Application Database to reach Pending phase..." kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.applicationDatabase.phase}'=Pending opsmanager/om --timeout=30s echo "Waiting for Ops Manager to reach Pending phase..." kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.opsManager.phase}'=Pending opsmanager/om --timeout=600s
Waiting for Application Database to reach Pending phase... mongodbopsmanager.mongodb.com/om condition met Waiting for Ops Manager to reach Pending phase... mongodbopsmanager.mongodb.com/om condition met
8
Configura servicios de balanceador de carga.
svcneg0=$(kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get svcneg -o=jsonpath='{.items[0].metadata.name}') kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" \ wait svcneg "${svcneg0}" --for=condition=Initialized --timeout=300s \ || { kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get svcneg "${svcneg0}" -o yaml; exit 1; } gcloud_retry compute backend-services add-backend "${OM_BACKEND_SERVICE_NAME}" \ --global \ --network-endpoint-group="${svcneg0}" \ --network-endpoint-group-zone="${K8S_CLUSTER_0_ZONE}" \ --balancing-mode RATE --max-rate-per-endpoint 5
svcneg1=$(kubectl --context "${K8S_CLUSTER_1_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get svcneg -o=jsonpath='{.items[0].metadata.name}') kubectl --context "${K8S_CLUSTER_1_CONTEXT_NAME}" -n "${OM_NAMESPACE}" \ wait svcneg "${svcneg1}" --for=condition=Initialized --timeout=300s \ || { kubectl --context "${K8S_CLUSTER_1_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get svcneg "${svcneg1}" -o yaml; exit 1; } gcloud_retry compute backend-services add-backend "${OM_BACKEND_SERVICE_NAME}" \ --global \ --network-endpoint-group="${svcneg1}" \ --network-endpoint-group-zone="${K8S_CLUSTER_1_ZONE}" \ --balancing-mode RATE --max-rate-per-endpoint 5
9
Espera a que Ops Manager entre en un estado de ejecución.
echo "Waiting for Application Database to reach Running phase..." kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.applicationDatabase.phase}'=Running opsmanager/om --timeout=1200s echo; echo "Waiting for Ops Manager to reach Running phase..." kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.opsManager.phase}'=Running opsmanager/om --timeout=1200s echo; echo "MongoDBOpsManager resource" kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get opsmanager/om echo; echo "Pods running in cluster ${K8S_CLUSTER_0_CONTEXT_NAME}" kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get pods echo; echo "Pods running in cluster ${K8S_CLUSTER_1_CONTEXT_NAME}" kubectl --context "${K8S_CLUSTER_1_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get pods
Waiting for Application Database to reach Running phase... mongodbopsmanager.mongodb.com/om condition met Waiting for Ops Manager to reach Running phase... mongodbopsmanager.mongodb.com/om condition met MongoDBOpsManager resource NAME REPLICAS VERSION STATE (OPSMANAGER) STATE (APPDB) STATE (BACKUP) AGE WARNINGS om 8.0.5 Running Running Disabled 18m Pods running in cluster gke_scratch-kubernetes-team_europe-central2-a_k8s-mdb-0-6aa01fd8df4x72f980-18899 NAME READY STATUS RESTARTS AGE om-0-0 1/1 Running 0 15m om-db-0-0 2/2 Running 0 5m17s om-db-0-1 2/2 Running 0 6m19s Pods running in cluster gke_scratch-kubernetes-team_europe-central2-b_k8s-mdb-1-6aa01fd8df4x72f980-18899 NAME READY STATUS RESTARTS AGE om-1-0 1/1 Running 0 15m om-1-1 1/1 Running 0 10m om-db-1-0 2/2 Running 0 3m31s om-db-1-1 2/2 Running 0 4m24s
10
Instale Minio.
kubectl kustomize "github.com/minio/operator/resources/?timeout=120&ref=v5.0.12" | \ kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" apply -f - kubectl kustomize "github.com/minio/operator/examples/kustomization/tenant-tiny?timeout=120&ref=v5.0.12" | \ kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" apply -f - add two buckets to the tenant config kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "tenant-tiny" patch tenant/myminio \ --type='json' \ -p="[{\"op\": \"add\", \"path\": \"/spec/buckets\", \"value\": [{\"name\": \"${S3_OPLOG_BUCKET_NAME}\"}, {\"name\": \"${S3_SNAPSHOT_BUCKET_NAME}\"}]}]"
11
Configure los secretos de Kubernetes para las copias de seguridad de Ops Manager.
kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" create secret generic s3-access-secret \ --from-literal=accessKey="${S3_ACCESS_KEY}" \ --from-literal=secretKey="${S3_SECRET_KEY}" minio TLS secrets are signed with the default k8s root CA kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" create secret generic s3-ca-cert \ --from-literal=ca.crt="$(kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n kube-system get configmap kube-root-ca.crt -o jsonpath="{.data.ca\.crt}")"
12
Habilite las copias de seguridad S3 (Minio) en Ops Manager.
kubectl apply --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" -f - <<EOF apiVersion: mongodb.com/v1 kind: MongoDBOpsManager metadata: name: om spec: topology: MultiCluster version: "${OPS_MANAGER_VERSION}" adminCredentials: om-admin-user-credentials externalConnectivity: type: ClusterIP annotations: cloud.google.com/neg: '{"exposed_ports": {"8443":{}}}' opsManagerURL: "https://${OPS_MANAGER_EXTERNAL_DOMAIN}" security: certsSecretPrefix: cert-prefix tls: ca: ca-issuer clusterSpecList: - clusterName: "${K8S_CLUSTER_0_CONTEXT_NAME}" members: 1 backup: members: 0 - clusterName: "${K8S_CLUSTER_1_CONTEXT_NAME}" members: 2 backup: members: 0 - clusterName: "${K8S_CLUSTER_2_CONTEXT_NAME}" members: 0 backup: members: 1 applicationDatabase: version: "${APPDB_VERSION}" topology: MultiCluster security: certsSecretPrefix: cert-prefix tls: ca: ca-issuer clusterSpecList: - clusterName: "${K8S_CLUSTER_0_CONTEXT_NAME}" members: 2 externalAccess: externalDomain: "${APPDB_CLUSTER_0_EXTERNAL_DOMAIN}" externalService: annotations: external-dns.alpha.kubernetes.io/hostname: "{podName}.${APPDB_CLUSTER_0_EXTERNAL_DOMAIN}" - clusterName: "${K8S_CLUSTER_1_CONTEXT_NAME}" members: 2 externalAccess: externalDomain: "${APPDB_CLUSTER_1_EXTERNAL_DOMAIN}" externalService: annotations: external-dns.alpha.kubernetes.io/hostname: "{podName}.${APPDB_CLUSTER_1_EXTERNAL_DOMAIN}" - clusterName: "${K8S_CLUSTER_2_CONTEXT_NAME}" members: 1 externalAccess: externalDomain: "${APPDB_CLUSTER_2_EXTERNAL_DOMAIN}" externalService: annotations: external-dns.alpha.kubernetes.io/hostname: "{podName}.${APPDB_CLUSTER_2_EXTERNAL_DOMAIN}" backup: enabled: true s3Stores: - name: my-s3-block-store s3SecretRef: name: "s3-access-secret" pathStyleAccessEnabled: true s3BucketEndpoint: "${S3_ENDPOINT}" s3BucketName: "${S3_SNAPSHOT_BUCKET_NAME}" customCertificateSecretRefs: - name: s3-ca-cert key: ca.crt s3OpLogStores: - name: my-s3-oplog-store s3SecretRef: name: "s3-access-secret" s3BucketEndpoint: "${S3_ENDPOINT}" s3BucketName: "${S3_OPLOG_BUCKET_NAME}" pathStyleAccessEnabled: true customCertificateSecretRefs: - name: s3-ca-cert key: ca.crt EOF
13
Espera a que Ops Manager entre en un estado de ejecución.
echo; echo "Waiting for Backup to reach Running phase..." kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.backup.phase}'=Running opsmanager/om --timeout=1200s echo "Waiting for Application Database to reach Running phase..." kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.applicationDatabase.phase}'=Running opsmanager/om --timeout=1200s echo; echo "Waiting for Ops Manager to reach Running phase..." kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" wait --for=jsonpath='{.status.opsManager.phase}'=Running opsmanager/om --timeout=1200s echo; echo "MongoDBOpsManager resource" kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get opsmanager/om echo; echo "Pods running in cluster ${K8S_CLUSTER_0_CONTEXT_NAME}" kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get pods echo; echo "Pods running in cluster ${K8S_CLUSTER_1_CONTEXT_NAME}" kubectl --context "${K8S_CLUSTER_1_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get pods echo; echo "Pods running in cluster ${K8S_CLUSTER_2_CONTEXT_NAME}" kubectl --context "${K8S_CLUSTER_2_CONTEXT_NAME}" -n "${OM_NAMESPACE}" get pods
Waiting for Backup to reach Running phase... mongodbopsmanager.mongodb.com/om condition met Waiting for Application Database to reach Running phase... mongodbopsmanager.mongodb.com/om condition met Waiting for Ops Manager to reach Running phase... mongodbopsmanager.mongodb.com/om condition met MongoDBOpsManager resource NAME REPLICAS VERSION STATE (OPSMANAGER) STATE (APPDB) STATE (BACKUP) AGE WARNINGS om 8.0.5 Running Running Running 23m Pods running in cluster gke_scratch-kubernetes-team_europe-central2-a_k8s-mdb-0-6aa01fd8df4x72f980-18899 NAME READY STATUS RESTARTS AGE om-0-0 1/1 Running 0 20m om-db-0-0 2/2 Running 0 10m om-db-0-1 2/2 Running 0 11m Pods running in cluster gke_scratch-kubernetes-team_europe-central2-b_k8s-mdb-1-6aa01fd8df4x72f980-18899 NAME READY STATUS RESTARTS AGE om-1-0 1/1 Running 0 20m om-1-1 1/1 Running 0 15m om-db-1-0 2/2 Running 0 8m41s om-db-1-1 2/2 Running 0 9m34s Pods running in cluster gke_scratch-kubernetes-team_europe-central2-c_k8s-mdb-2-6aa01fd8df4x72f980-18899 NAME READY STATUS RESTARTS AGE om-2-backup-daemon-0 1/1 Running 0 4m45s om-db-2-0 2/2 Running 0 7m37s
14
Crea una organización de MongoDB y obtén credenciales.
Para configurar las credenciales, debe crear una organización en el Ops Manager, generar claves API programáticas en la Interfaz de Usuario del Ops Manager, y crear un secreto con la IP de su balanceador de carga. Consulte Crear Credenciales para el Operador de Kubernetes para aprender más.