多集群架构图： MongoDB Ops Manager和应用程序数据库

下图显示了部署在多个Kubernetes集群上的MongoDB Ops Manager应用程序、应用程序数据库、备份守护程序和相应的持久卷。

在此图中：

1. Member Cluster 0也是一个“操作符集群”，因为您在其上安装了Kubernetes Operator。 它也是一个“成员集群”，可以托管任何多集群自定义资源。

2. The Member Cluster 0 stores the kubeconfig files, which describe the Kubernetes configuration for member clusters, users, and contexts. When you configure the Kubernetes Operator for multi-cluster deployments using the kubectl mongodb plugin, it creates the following resource:

   • mongodb-enterprise-operator-multi-cluster-kubeconfig密钥，包含 Kubernetes Operator 将管理的所有 Kubernetes 集群的档案。 如果您计划将 Operator 集群用作成员集群，则此密钥可能包含您安装 Kubernetes Operator 的同一集群的档案。

   当 Kubernetes Operator 在多集群模式下运行时，它会存储所需的资源，例如 ConfigMap 和有关要管理的集群的密钥。 这些资源与 Kubernetes Operator 属于同一命名空间。 KubernetesOperator 使用这些资源在多个MongoDB Ops Manager Kubernetes集群上部署 应用程序和应用程序数据库。

3. Kubernetes Operator 还为其管理的每个MongoDB Ops Manager应用程序和应用程序数据库部署创建并维护一些额外的多集群部署状态 ConfigMap。 Member Cluster 0存储此配置，其中包括以下 ConfigMap：

   • <om_resource_name>-cluster-mapping ConfigMap 包含spec.clusterSpecList中列出的成员集群名称到集群索引的映射，在本文档中称为cluster_index ，例如Cluster 0或Cluster 1 。 Kubernetes Operator 将这些索引分配给每个集群名称。

   • <om_resource_name>-db-cluster-mapping ConfigMap 包含spec.applicationDatabase.clusterSpecList中列出的成员集群名称到集群索引的映射。

   • <om_resource_name>-db-member-spec ConfigMap 包含为每个成员集群配置的应用程序数据库副本数。 有了这些信息， Kubernetes Operator 就可以在灾难恢复中正确扩展或重新配置副本集，例如在丢失整个成员集群后。

4. MongoDBOpsManager 资源的配置是您创建的一个文件，用于描述多集群MongoDB Ops Manager部署。 Kubernetes Operator 使用此文件来部署MongoDB Ops Manager组件。

   以下示例显示了导致Kubernetes Operator 部署此图中描述的MongoDB Ops Manager组件的配置。 此示例省略了一些与此图表不相关的设置，例如TLS配置。

   1
   apiVersion: mongodb.com/v1
   2
   kind: MongoDBOpsManager
   3
   metadata:
   4
     name: om
   5
     namespace: om-ns
   6
   spec:
   7
     replicas: 1 # You can set this value and use it as a global or default
   8
                 # setting for all clusters. The spec.clusterSpecList.members
   9
                 # setting overrides this setting.
   10
     topology: MultiCluster
   11
     version: 8.0.0
   12
     adminCredentials: om-admin-secret
   13
     clusterSpecList:
   14
       - clusterName: "Member Cluster 1" # Ops Manager settings for "Member Cluster 1"
   15
         members: 2
   16
         backup: # Backup settings for "Member Cluster 1"
   17
           members: 2 # Overrides spec.backup.members
   18
       - clusterName: "Member Cluster 2" # Ops Manager settings for "Member Cluster 2"
   19
         members: 1
   20
         backup: # Backup settings for "Member Cluster 2"
   21
           members: 2 # Overrides spec.backup.members
   22
     applicationDatabase: # Global {+appdb+} settings
   23
       topology: MultiCluster
   24
       version: 8.0.0
   25
       members: 3 # In multi-cluster mode, the Operator ignores this field.
   26
                  # The Operator sets the number of members for the Application
   27
                  # Database in spec.applicationDatabase.clusterSpecList.members.
   28
       clusterSpecList:
   29
         - clusterName: "Member Cluster 1"
   30
           members: 3
   31
         - clusterName: "Member Cluster 2"
   32
           members: 2
   33
     backup: # Global settings for the Backup Daemon
   34
       enabled: true
   35
       members: 1 # Set this value and use it as a global or default setting.
   36
                  # To override this value, set the value for
   37
                  # spec.clusterSpecList.backup.members.
   38
                  # The Backup Daemon's configuration for each cluster isn't
   39
                  # stored here. Use the Ops Manager's spec.clusterSpecList.backup to
   40
                  # specify the Backup Daemon configuration for each member cluster.

5. Kubernetes Operator 连接到引用以下任一项的MongoDB Ops Manager实例：

   • The default FQDN of the service it creates for the Ops Manager resource, <om_resource_name>-svc.<namespace>.svc.cluster.local, or

   • The URL that you specify in spec.opsManagerURL. In some deployments, such as when the cluster where you installed the Kubernetes Operator isn't attached to the service mesh, the default service FQDN might be unreachable. In this case, the Kubernetes Operator reports the MongoDBOpsManager resource status as Failed indicating a connection error. To account for such cases, provide the URL to Ops Manager in the spec.opsManagerURL. This URL might be a hostname of an externally exposed Ops Manager instance. To learn more, see Networking Overview.

6. Two member clusters host the Ops Manager Application. In each cluster, the Kubernetes Operator deploys a StatefulSet named <om_resource_name>-<cluster_index>.

   • StatefulSet 在 Member Cluster 1 中部署MongoDB Ops Manager应用程序的两个实例，在 Member Cluster 2 中部署一个实例。

   • 您在spec.clusterSpecList.members中定义实例的数量。 您可以设立实例数设置为零，以便此集群不部署任何MongoDB Ops Manager应用程序实例。 示例，如果您想使用此集群仅托管备份守护程序实例，这非常有用。

     如果从spec.clusterSpecList删除一个集群，则相当于在spec.clusterSpecList.members和spec.clusterSpecList[*].backup.members中指定零个成员。

   • For each StatefulSet in each cluster, the Kubernetes Operator configures a service of type ClusterIP, named <om_resource_name>-svc, that contains all Pods on the cluster's endpoints list. This service's FQDN, <om_resource_name>-svc.<namespace>.svc.cluster.local, is a default hostname that the Kubernetes Operator uses to access the deployed endpoint for the Ops Manager Application.

   • 如果指定spec.externalConnectivity ， Kubernetes Operator 还会为每个集群创建一个名为<om_resource_name>-svc-ext的外部Kubernetes LoadBalancer类型服务。 在每个集群中，您可以使用spec.clusterSpecList.externalConnectivity为此外部服务指定自己的配置。 示例，您可以更改服务类型或定义注解。

7. 应用程序数据库。 Kubernetes Operator 在两个集群上部署应用程序数据库。

   • The Member Cluster 1 contains three mongod processes for the Application Database, and the Member Cluster 2 contains two mongod processes..

   • You define the Application Database configuration using the spec.applicationDatabase settings. On each member cluster, the Kubernetes Operator creates a StatefulSet named <om_resource_name>-db-<cluster_index> with the number of member clusters defined in spec.applicationDatabase.clusterSpecList.members. In multi-cluster mode, the Kubernetes Operator ignores values that you set for the spec.applicationDatabase.members field. The Kubernetes Operator configures one replica set formed from mongod processes deployed across all member clusters.

   • For each Pod in <statefulset_name>-<pod_index> hosting a MongoDB process named <om_resource_name>-db-<cluster_index>-<pod_index>, the Kubernetes Operator creates a Kubernetes ClusterIP-type service for accessing the individual mongod processes by its FQDN, <om_resource_name>-db-<cluster_index>-<pod_index>-svc. Each mongod process in the replica set must be uniquely addressable.

     The processes in the replica set configuration must have their process hostnames configured to that Pod service's FQDN: <om_resource_name>-db-<cluster_index>-<pod_index>-svc.<namespace>.svc.cluster.local.

   • 每个 Pod 都通过Kubernetes Operator 创建的持久卷声明附加了持久卷。

   • To form a replica set from all mongod processes, each process must connect to each other process for replication purposes. To achieve this, include all member clusters on which you deploy the Application Database into the same service mesh configuration.

     服务网格处理跨集群 DNS 查询并相应地路由流量。服务网格协助解析所有集群中每个 Pod 服务的 FQDN <om_resource_name>-db-<cluster_index>-<pod-index>-svc.<namespace>.svc.cluster.local，并允许在公开的 mongod 端口（默认为 27017）上进行连接。

     For example, when a mongod process running in the om-db-1-0 Pod in Member Cluster 1 connects to a mongod running in the om-db-2-1 Pod in Member Cluster 2, the first mongod process uses its hostname from the Automation Configuration, om-db-2-1-svc.om-ns.svc.cluster.local:27017, and the service mesh routes this request to Member Cluster 2 to the om-db-2-1-svc service. Without the service mesh, the Kubernetes Member Cluster 1 has no information about the om-db-2-1-svc service deployed in the Member Cluster 2 and the DNS resolution of om-db-2-1-svc.om-ns.svc.cluster.local would fail.

   • 当应用程序数据库和MongoDB Ops Manager应用程序实例处于 Running 状态时， Kubernetes Operator 会向应用程序数据库 StatefulSets 添加一个额外的监控容器。 这会导致所有集群中的所有应用程序数据库 Pod滚动重启。 Kubernetes Operator 按顺序更新所有集群的 StatefulSet，以便在滚动重启进程，每个集群中只有一个副本集的成员暂时不可用。

   • The Monitoring Agent connects to the Ops Manager Application instances using the Ops Manager service's FQDN, <om_resource_name>-svc.<namespace>.svc.cluster.local, or the value in spec.opsManagerURL if you specify it.

     MongoDB Ops Manager应用程序和备份守护程序始终使用包含所有副本集成员的应用程序数据库的连接string 。 连接string始终使用 pod 服务 FQDN 构建。

8. 如果您将spec.backup.enabled设置为true ，Kubernetes Operator 会部署备份守护程序 StatefulSet。

   • 在spec.clusterSpecList中列出的每个成员集群上，Kubernetes Operator 都会创建一个备份守护程序 StatefulSet，名为<om_resource_name>-backup-daemon-<cluster_index> ，并将备份守护程序实例的数量设置为spec.backup.members 。

     或者，您可以在spec.clusterSpecList[*].backup.members中配置每个集群的备份守护程序实例的数量。

   • 备份守护程序实例仅使用与 应用程序实例相同的连接string MongoDB Ops Manager连接到应用程序数据库副本集。

此外，在此图中，您可以观察服务网格和组件之间的网络连接：

• 该图周围的虚线显示了单个服务网格，其中包括所有集群的网络配置。

• 跨成员集群的MongoDB Ops Manager应用程序周围的虚线表示这些实例是无状态的，并且流量可以平均分布式给所有实例，示例使用循环负载负载均衡器。

• 跨成员集群的应用程序数据库周围的虚线表示这些实例相互通信并形成单个 MongoDB 副本集。