您可以将MongoDB分片集群分布在多个Kubernetes集群上。通过多集群功能,您可以:
通过将部署分布在多个Kubernetes集群(每个集群位于不同的地理地区)中,提高部署的韧性。
通过在不同Kubernetes集群中部署指定分片的主节点 (primary node in the replica set)节点来配置地理分片的部署,这些集群距离依赖于该数据的应用程序或客户端更近,从而减少延迟。
调整部署以提高性能。示例,您可以为不同Kubernetes集群中的所有分片或指定分片部署只读分析节点,也可以自定义资源分配。
此外,您可以在不同级别配置分片、 mongos和配置服务器详细信息。这意味着您可以为分片、配置服务器和mongos定义顶级默认配置,并为每个Kubernetes集群独立进行自定义。此外,还可以自定义单个分片以满足您的特定需求。
重要
多集群分片分片集群功能使得在多个地理区域的多个Kubernetes集群上部署MongoDB资源成为可能;但是,这样做可能会增加复制操作的延迟。
要学习;了解有关多集群分片集群拓扑结构的特定配置选项的更多信息,请参阅分片集群参考。
限制
Hashicorp Vault支持不适用于Kubernetes密钥注入。
如果成员集群发生故障,必须首先手动扩展故障集群缩减至零,以删除运行状况不佳的进程。然后,您可以通过更新部署到中央集群的CRD资源,手动在剩余健康Kubernetes集群之间重新分配资源。请参阅灾难恢复以学习;了解更多信息。
先决条件
为多集群部署准备Kubernetes集群。
安装并配置Kubernetes Operator,以便在其中一个Kubernetes集群中进行多集群部署。
选项1 :跨成员集群部署服务网格。您可以在《MongoDB Ops Manager多集群部署指南》的步骤 1-6 中找到详细过程。请注意,此过程是固定的,并且只是许多可能配置中的一种。
选项 2:为非服务网格部署配置Kubernetes集群。
将包含MongoDB Ops Manager或Cloud Manager以及MongoDB项目详细信息的配置映射部署到中央集群, Kubernetes Operator 需要该配置映射来部署MongoDB 数据库资源。
分片集群部署示例
当应用于中央集群时,以下示例部署了一个包含 3 分片的分片MongoDB 集群,配置如下:
每个分片都有分布式在所有Kubernetes集群的节点(每个集群1 个节点)。
默认下,每个分片的副本集:
在 3 个集群中总共部署了 3 个有投票权的成员(每个集群中有 1 个节点)
主分片最好位于
kind-e2e-cluster-1中。
通过分片
sc-0的shardOverride,我们将默认值(在spec.shard中指定)更改为以下内容:总共 5 个成员
2 成员位于
kind-e2e-cluster-12 成员位于
kind-e2e-cluster-21 成员
kind-e2e-cluster-3如果可能,主分片最好位于
kind-e2e-cluster-2
为所有集群中的所有分片自定义默认存储设置,如
spec.shardPodSpec具有 3 个成员的配置服务器副本集,每个集群具有 1 个成员
总共部署 3 个mongos实例,每个集群有 1 个实例
默认配置由三个分片组成,每个分片在三个集群上各有一个成员,每个分片总共有三个成员。但在覆盖中,我们将分片sc-0 更改为具有五个成员,其中两个在集群1 上,两个在集群2 上,并且集群3 仍然具有默认的一个分片。
注意
仅支持单向扩展。
当您扩展分片、配置服务器或mongos副本时,您只能向上或向下扩展资源,以确保扩展的正确性。此规则适用于单个Kubernetes集群部署和分布式在多个Kubernetes集群的部署。此外,在多集群部署中,单向扩展规则适用于所有资源类型。示例,您不能在删除(扩展)配置服务器或mongos 的同时向分片添加更多节点(向上扩展)。在不更改成员总数的情况下,您无法再将一个节点从一个集群“移动”到另一个集群。相反,您必须首先执行纵向扩展操作,然后执行单独的扩展更改。
对于分片sc-0,此示例配置还会(使用 shardOverrides)将主节点 (primary node in the replica set)转移到集群2,这可以减少在集群2 所在地区中操作的用户的延迟。这样,您仍然具有跨集群(及其区域)的韧性,但如果分片0 中的数据是与部署集群2 的用户最相关的数据,则他们将体验到更低的延迟。
apiVersion: mongodb.com/v1kind: MongoDBmetadata: name: scspec: topology: MultiCluster type: ShardedCluster # this deployment will have 3 shards shardCount: 3 # you cannot specify mongodsPerShardCount, configServerCount and mongosCount # in MultiCluster topology version: 8.0.3 opsManager: configMapRef: name: my-project credentials: my-credentials persistent: true
shardPodSpec: # applies to all shards on all clusters persistence: single: # all pods for all shards on all clusters will use that storage size in their # PersistentVolumeClaim unless overridden in spec.shard.clusterSpecList or # spec.shardOverrides. storage: 10G
configSrvPodSpec: # applies to all config server nodes in all clusters persistence: multiple: data: storage: 2G journal: storage: 1G logs: storage: 1G
# consider this section as a default configuration for ALL shards shard: clusterSpecList: - clusterName: kind-e2e-cluster-1 # each shard will have only one mongod process deployed in this cluster members: 1 memberConfig: - votes: 1 priority: "20" # we increase the priority to have primary in this cluster - clusterName: kind-e2e-cluster-2 # one member in this cluster, no votes and priority defined means it'll get # the default values votes=1, priority="1" members: 1 - clusterName: kind-e2e-cluster-3 members: 1 # one member in this cluster
shardOverrides: # here you specify customizations for specific shards # here you specify to which shard names the following configuration will # apply - shardNames: - sc-0 clusterSpecList: - clusterName: kind-e2e-cluster-1 # all fields here are optional # shard "sc-0" will have two members instead of one, which was defined as the # default for all shards in spec.shard.clusterSpecList[0].members members: 2 memberConfig: - votes: 1 # shard "sc-0" should not have primary in this cluster like every other shard priority: "1" - votes: 1 priority: "1" - clusterName: kind-e2e-cluster-2 members: 2 # shard "sc-0" will have two members instead of one memberConfig: - votes: 1 # both processes of shard "sc-0" in this cluster will have the same # likelihood to become a primary member priority: "20" - votes: 1 priority: "20" # We need to specify the list of all clusters on which this shard will be # deployed. - clusterName: kind-e2e-cluster-3 # If the clusterName element is omitted here, it will be considered as an # override for this shard, so that the operator shouldn't deploy any member # to it. # No fields are mandatory in here, though. In case a field is not set, it's # not overridden and the default value is taken from a top level spec.shard # settings.
configSrv: # the same configuration fields are available as in # spec.shard.clusterSpecList. clusterSpecList: - clusterName: kind-e2e-cluster-1 members: 1 - clusterName: kind-e2e-cluster-2 members: 1 - clusterName: kind-e2e-cluster-3 members: 1
mongos: # the same configuration fields are available as in # spec.shard.clusterSpecList apart from storage and replica-set related # fields. clusterSpecList: - clusterName: kind-e2e-cluster-1 members: 1 - clusterName: kind-e2e-cluster-2 members: 1 - clusterName: kind-e2e-cluster-3 members: 1分片覆盖
要将分片集群部署到多个 Kubernetes 集群,请将分片集群配置(MongoDB 自定义资源 yaml)应用到您的中央集群,即部署 MongoDB Operator 实例的 Kubernetes 集群。此配置的 spec.shard 定义被视为部署的基本分片定义。
如果要在特定Kubernetes集群上自定义特定分片,可以使用分片覆盖来更新给定分片的基本分片定义。
下表列出了您可以定义以更新或扩展基本分片定义的字段。字段按优先顺序列出。给定表中的最顶层字段表示优先级最低的设置,最底层字段(如果已定义)将覆盖所有其他字段(例如) 。
此外,为每种字段类型表示的覆盖策略描述了该特定字段是否被新定义的值覆盖,或者定义该字段的完整对象是否被覆盖。如果覆盖了整个对象,则在基本分片定义中定义且未在覆盖定义中显式定义的任何字段都将被删除。merge 值表示更新单个字段,replace 值表示更新单个字段。完整的父对象被覆盖。
自定义 Persistence 和 Statefulset 设置
ShardedClusterSpec 字段 | 要指定 | 适用于 |
|---|---|---|
| 持久性和 Pod 模板 | 所有 Pod |
| 持久性和 Pod 模板 | 所有 Pod |
| 持久性(忽略 Pod 模板字段) | 集群中的所有分片 |
| Pod Template | 集群中的所有分片 |
| 持久性(忽略 Pod 模板字段) | 集群中的所有分片 |
| Pod Template | 集群中的一个分片 |
| 持久性(忽略 Pod 模板字段) | 特定集群中的一个分片 |
| Pod Template | 特定集群中的一个分片 |
注意
弃用 ShardSpecificPodSpec
ShardSpecificPodSpec字段已弃用,但仍受支持。它以前用于为单个集群分片集群指定每个分片的持久性和 Pod 模板参数。 现在它已弃用,您应该迁移到 ShardOverrides.PodSpec 和 ShardOverrides.StatefulSetConfiguration。有关从单集群部署的 shardSpecificPodSpec 和 shardOverrides 进行更新的指导,请参阅提供的示例YAML文件。
覆盖 Pod 模板和持久性设置 clusterSpecList
以下示例说明了如何覆盖 clusterSpecList 中的自定义 Pod 模板和持久性设置。
shardPodSpec: # applicable to all shards in all clusters persistence: single: storage: "5G" podTemplate: spec: containers: - name: mongodb-enterprise-database resources: requests: cpu: 0.5 memory: 1.0G limits: cpu: 1.0 memory: 2.0G shard: clusterSpecList: - clusterName: kind-e2e-cluster-1 members: 2 # The below statefulset override is applicable only to pods in kind-e2e-cluster-1 # Specs will be merged, the "request" field defined above will still be # applied to containers in this cluster. # However, limits will be replaced with below values, because # clusterSpecList.statefulSet.spec.template has a # higher priority than shardPodSpec.podTemplate statefulSet: spec: template: spec: containers: - name: mongodb-enterprise-database resources: limits: cpu: 1.0 memory: 2.5G # In clusterSpecList.podSpec, only persistence field must be used, the # podTemplate field is ignored. # In kind-e2e-cluster-1, we replace the persistence settings defined in # shardPodSpec podSpec: persistence: multiple: journal: storage: "6G" data: storage: "7G" logs: storage: "6G" - clusterName: kind-e2e-cluster-2 members: 1要学习;了解更多信息,您可以查看完整文件。
定义自定义 Pod 模板和持久性设置 shardOverrides
以下示例说明了如何在 shardOverrides 中定义自定义 Pod 模板和持久性设置。
# In clusterSpecList.podSpec, only persistence field must be used, the # podTemplate field is ignored. # In kind-e2e-cluster-1, we define custom persistence settings podSpec: persistence: multiple: journal: storage: "5G" data: storage: "5G" logs: storage: "5G" - clusterName: kind-e2e-cluster-2 members: 1
shardOverrides: - shardNames: [ "pod-template-shards-1-2" ] # This override will apply to shard of index 2 # Statefulset settings defined at this level (shardOverrides.statefulSet) # apply to members of shard 2 in ALL clusters. # This field has higher priority than shard.clusterSpecList.statefulSet, but # lower than shardOverrides.clusterSpecList.statefulSet # It has a merge policy, which means that the limits defined above for the # mongodb-enterprise-database container field still apply to all members in # that shard, except if overridden. statefulSet: spec: template: spec: containers: - name: sidecar-shard-2 image: busybox command: [ "sleep" ] args: [ "infinity" ] clusterSpecList: - clusterName: kind-e2e-cluster-1 members: 2 - clusterName: kind-e2e-cluster-2 members: 1 # The below statefulset override is applicable only to members of shard 2, in cluster 1 # Specs will be merged, the "limits" field defined above will still be applied # to containers in this cluster together with the requests field below. statefulSet: spec: template: spec: containers: - name: mongodb-enterprise-database resources: requests: # We add a requests field in shard 2, cluster 1 cpu: 0.5 memory: 1.0G
podSpec: # In shardOverrides.clusterSpecList.podSpec, only persistence field must be # used, the podTemplate field is ignored. persistence: # we assign additional disk resources in shard 2, cluster 1 multiple: journal: storage: "6G" data: storage: "6G" logs: storage: "6G"要学习;了解更多信息,您可以查看完整文件。
从已弃用的 shardSpecificPodSpec字段迁移
以下示例说明了如何从已弃用的 shardSpecificPodSpec字段更新到新的 shardOverrides字段。
# This file is an example of how to migrate from the old deprecated# ShardSpecificPodSpec field to the new shardOverrides fields# for single cluster deployments.# The settings specified in shardOverrides are the exact equivalent to the# ones in shardSpecificPodSpec, showing how to replicate themapiVersion: mongodb.com/v1kind: MongoDBmetadata: name: shardspecificpodspec-migration namespace: mongodb-testspec: # There are 4 shards in this cluster, but the shardSpecificPodSpec field # doesn't need to have on entry per shard, it can have less shardCount: 4 mongodsPerShardCount: 2 mongosCount: 1 configServerCount: 3 topology: SingleCluster type: ShardedCluster version: 8.0.3 opsManager: configMapRef: name: my-project credentials: my-credentials persistent: true
shardPodSpec: # default persistence configuration for all shards in all clusters persistence: single: storage: "5G" shardSpecificPodSpec: # deprecated way of overriding shards (array) - persistence: # shard of index 0 single: storage: "6G" # Specify resources settings to enterprise database container in shard 0 podTemplate: spec: containers: - name: mongodb-enterprise-database resources: requests: cpu: 0.5 memory: 1G limits: cpu: 1.0 memory: 2.0G - persistence: # shard of index 1 single: storage: "7G" - persistence: # shard of index 2 single: storage: "7G"
# The below shardOverrides replicate the same shards configuration as the one # specified above in shardSpecificPodSpec shardOverrides: - shardNames: [ "shardspecificpodspec-migration-0" ] # overriding shard #0 podSpec: persistence: single: storage: "6G" statefulSet: spec: template: spec: containers: - name: mongodb-enterprise-database resources: requests: cpu: 0.5 memory: 1G limits: cpu: 1.0 memory: 2.0G
# The ShardSpecificPodSpec field above has the same configuration for shards # 1 and 2. It is possible to specify both shard names in the override and not # duplicate that configuration - shardNames: [ "shardspecificpodspec-migration-1", "shardspecificpodspec-migration-2" ] podSpec: persistence: single: storage: "7G"要学习;了解更多信息,您可以查看完整文件。
外部访问配置
字段 | 哪些集群 | 哪些分片 |
|---|---|---|
| 所有 | 所有 |
| one | 所有 |
| one | 所有 |
Members、MemberConfig、Additional MongodConfig、AgentConfig
ShardedClusterSpec 字段 | 要指定 | 适用于 |
|---|---|---|
| 不适用 | 不适用 |
| 适用于 | 适用于 |
| 适用于 | 适用于 |
| 不适用 | 不适用 |
| 适用于 | 适用于 |
配置服务器覆盖
ShardedClusterSpec 字段 | 要指定 | 适用于 |
|---|---|---|
| 不适用 | 不适用 |
| 适用于 | 适用于 |
| 不适用 | 不适用 |
| 适用于 | 忽略 |
| 不适用 | 适用于 |
configSrv: clusterSpecList: - clusterName: kind-e2e-cluster-1 members: 2 # The below statefulset override is applicable only to pods in kind-e2e-cluster-1 # Specs will be merged, the "request" field defined above will still be applied to containers in this cluster # However, limits will be replaced with below values, because clusterSpecList.statefulSet.spec.template has a # higher priority than configSrvPodSpec.podTemplate statefulSet: spec: template: spec: containers: - name: mongodb-enterprise-database resources: limits: cpu: 1.0 memory: 2.5G # In clusterSpecList.podSpec, only persistence field must be used, the podTemplate field is ignored. podSpec: # In kind-e2e-cluster-1, we replace the persistence settings defined in configSrvPodSpec persistence: multiple: journal: storage: "6G" data: storage: "7G" logs: storage: "6G" - clusterName: kind-e2e-cluster-2 members: 1 # doc-highlight-end: configSrv mongos: clusterSpecList: - clusterName: kind-e2e-cluster-1 members: 2 - clusterName: kind-e2e-cluster-2 members: 1
shard: clusterSpecList: - clusterName: kind-e2e-cluster-1 members: 2 - clusterName: kind-e2e-cluster-2 members: 1要学习;了解更多信息,您可以查看完整文件。
Mongos 覆盖
ShardedClusterSpec 字段 | 要指定 | 适用于 |
|---|---|---|
| 不适用 | 不适用 |
| 适用于 | 不适用 |
| 不适用 | 适用于 |
| 忽略 | 不适用 |
| 适用于 | 不适用 |