多集群分片集群
在此页面上
您可以将MongoDB分片集群分布在多个Kubernetes集群上。通过多集群功能,您可以:
通过将部署分布在多个Kubernetes集群(每个集群位于不同的地理地区)中,提高部署的韧性。
通过在不同Kubernetes集群中部署指定分片的主节点 (primary node in the replica set)节点来配置地理分片的部署,这些集群距离依赖于该数据的应用程序或客户端更近,从而减少延迟。
调整部署以提高性能。示例,您可以为不同Kubernetes集群中的所有分片或指定分片部署只读分析节点,也可以自定义资源分配。
此外,您可以在不同级别配置分片、 mongos和配置服务器详细信息。这意味着您可以为分片、配置服务器和mongos定义顶级默认配置,并为每个Kubernetes集群独立进行自定义。此外,还可以自定义单个分片以满足您的特定需求。
重要
多集群分片分片集群功能使得在多个地理区域的多个Kubernetes集群上部署MongoDB资源成为可能;但是,这样做可能会增加复制操作的延迟。
要学习;了解有关多集群分片集群拓扑结构的特定配置选项的更多信息,请参阅 分片集群参考。
限制
Hashicorp Vault支持不适用于Kubernetes密钥注入。
Kubernetes Operator 不会自动将资源从出现故障的Kubernetes集群转移到正常集群。如果成员集群发生故障,您必须通过更新部署到中央集群的 CRD资源,手动在剩余健康Kubernetes集群之间重新分配资源。
如果成员集群发生故障,必须首先手动扩展故障集群缩减至零,以删除运行状况不佳的进程。然后,您可以通过更新部署到中央集群的CRD资源,手动在剩余健康Kubernetes集群之间重新分配资源。请参阅灾难恢复以学习;了解更多信息。
先决条件
为多集群部署准备Kubernetes集群。
安装并配置Kubernetes Operator,以便在其中一个Kubernetes集群中进行多集群部署。
选项1 :跨成员集群部署服务网格。您可以在《MongoDB Ops Manager多集群部署指南》的步骤 1-6 中找到详细过程。请注意,此过程是固定的,并且只是许多可能配置中的一种。
选项 2:为非服务网格部署配置Kubernetes集群。
将包含MongoDB Ops Manager或Cloud Manager以及MongoDB项目详细信息的配置映射部署到中央集群, Kubernetes Operator 需要该配置映射来部署MongoDB 数据库资源。
分片集群部署示例
当应用于中央集群时,以下示例部署了一个包含 3 分片的分片MongoDB 集群,配置如下:
每个分片都有分布式在所有Kubernetes集群的节点(每个集群1 个节点)。
默认下,每个分片的副本集:
在 3 个集群中总共部署了 3 个有投票权的成员(每个集群中有 1 个节点)
主分片最好位于
kind-e2e-cluster-1
中。
通过分片
sc-0
的shardOverride
,我们将默认值(在spec.shard
中指定)更改为以下内容:总共 5 个成员
2 成员位于
kind-e2e-cluster-1
2 成员位于
kind-e2e-cluster-2
1 成员
kind-e2e-cluster-3
如果可能,主分片最好位于
kind-e2e-cluster-2
为所有集群中的所有分片自定义默认存储设置,如
spec.shardPodSpec
具有 3 个成员的配置服务器副本集,每个集群具有 1 个成员
总共部署 3 个mongos实例,每个集群有 1 个实例
默认配置由三个分片组成,每个分片在三个集群上各有一个成员,每个分片总共有三个成员。但在覆盖中,我们将分片sc-0
更改为具有五个成员,其中两个在集群1
上,两个在集群2
上,并且集群3
仍然具有默认的一个分片。
注意
仅支持单向扩展。
当您扩展分片、配置服务器或mongos副本时,您只能向上或向下扩展资源,以确保扩展的正确性。此规则适用于单个Kubernetes集群部署和分布式在多个Kubernetes集群的部署。此外,在多集群部署中,单向扩展规则适用于所有资源类型。示例,您不能在删除(扩展)配置服务器或mongos 的同时向分片添加更多节点(向上扩展)。在不更改成员总数的情况下,您无法再将一个节点从一个集群“移动”到另一个集群。相反,您必须首先执行纵向扩展操作,然后执行单独的扩展更改。
对于分片sc-0
,此示例配置还会(使用 shardOverrides
)将主节点 (primary node in the replica set)转移到集群2
,这可以减少在集群2
所在地区中操作的用户的延迟。这样,您仍然具有跨集群(及其区域)的韧性,但如果分片0
中的数据是与部署集群2
的用户最相关的数据,则他们将体验到更低的延迟。
1 apiVersion: mongodb.com/v1 2 kind: MongoDB 3 metadata: 4 name: sc 5 spec: 6 topology: MultiCluster 7 type: ShardedCluster 8 # this deployment will have 3 shards 9 shardCount: 3 10 # you cannot specify mongodsPerShardCount, configServerCount and mongosCount 11 # in MultiCluster topology 12 version: 8.0.3 13 opsManager: 14 configMapRef: 15 name: my-project 16 credentials: my-credentials 17 persistent: true 18 19 shardPodSpec: # applies to all shards on all clusters 20 persistence: 21 single: 22 # all pods for all shards on all clusters will use that storage size in their 23 # PersistentVolumeClaim unless overridden in spec.shard.clusterSpecList or 24 # spec.shardOverrides. 25 storage: 10G 26 27 configSrvPodSpec: # applies to all config server nodes in all clusters 28 persistence: 29 multiple: 30 data: 31 storage: 2G 32 journal: 33 storage: 1G 34 logs: 35 storage: 1G 36 37 # consider this section as a default configuration for ALL shards 38 shard: 39 clusterSpecList: 40 - clusterName: kind-e2e-cluster-1 41 # each shard will have only one mongod process deployed in this cluster 42 members: 1 43 memberConfig: 44 - votes: 1 45 priority: "20" # we increase the priority to have primary in this cluster 46 - clusterName: kind-e2e-cluster-2 47 # one member in this cluster, no votes and priority defined means it'll get 48 # the default values votes=1, priority="1" 49 members: 1 50 - clusterName: kind-e2e-cluster-3 51 members: 1 # one member in this cluster 52 53 shardOverrides: # here you specify customizations for specific shards 54 # here you specify to which shard names the following configuration will 55 # apply 56 - shardNames: 57 - sc-0 58 clusterSpecList: 59 - clusterName: kind-e2e-cluster-1 60 # all fields here are optional 61 # shard "sc-0" will have two members instead of one, which was defined as the 62 # default for all shards in spec.shard.clusterSpecList[0].members 63 members: 2 64 memberConfig: 65 - votes: 1 66 # shard "sc-0" should not have primary in this cluster like every other shard 67 priority: "1" 68 - votes: 1 69 priority: "1" 70 - clusterName: kind-e2e-cluster-2 71 members: 2 # shard "sc-0" will have two members instead of one 72 memberConfig: 73 - votes: 1 74 # both processes of shard "sc-0" in this cluster will have the same 75 # likelihood to become a primary member 76 priority: "20" 77 - votes: 1 78 priority: "20" 79 # We need to specify the list of all clusters on which this shard will be 80 # deployed. 81 - clusterName: kind-e2e-cluster-3 82 # If the clusterName element is omitted here, it will be considered as an 83 # override for this shard, so that the operator shouldn't deploy any member 84 # to it. 85 # No fields are mandatory in here, though. In case a field is not set, it's 86 # not overridden and the default value is taken from a top level spec.shard 87 # settings. 88 89 configSrv: 90 # the same configuration fields are available as in 91 # spec.shard.clusterSpecList. 92 clusterSpecList: 93 - clusterName: kind-e2e-cluster-1 94 members: 1 95 - clusterName: kind-e2e-cluster-2 96 members: 1 97 - clusterName: kind-e2e-cluster-3 98 members: 1 99 100 mongos: 101 # the same configuration fields are available as in 102 # spec.shard.clusterSpecList apart from storage and replica-set related 103 # fields. 104 clusterSpecList: 105 - clusterName: kind-e2e-cluster-1 106 members: 1 107 - clusterName: kind-e2e-cluster-2 108 members: 1 109 - clusterName: kind-e2e-cluster-3 110 members: 1
分片覆盖
要将分片集群部署到多个Kubernetes集群,您应用分片集群配置(MongoDB自定义资源yaml)应用于操作符集群,即部署MongoDB Operator实例的Kubernetes集群。此配置的 spec.shard
定义被视为部署的基本分片定义。
如果要在特定Kubernetes集群上自定义特定分片,可以使用分片覆盖来更新给定分片的基本分片定义。
下表列出了您可以定义以更新或扩展基本分片定义的字段。字段按优先顺序列出。给定表中的最顶层字段表示优先级最低的设置,最底层字段(如果已定义)将覆盖所有其他字段(例如) 。
此外,为每种字段类型表示的覆盖策略描述了该特定字段是否被新定义的值覆盖,或者定义该字段的完整对象是否被覆盖。如果覆盖了整个对象,则在基本分片定义中定义且未在覆盖定义中显式定义的任何字段都将被删除。merge
值表示更新单个字段,replace
值表示更新单个字段。完整的父对象被覆盖。
自定义 Persistence 和 Statefulset 设置
ShardedClusterSpec 字段 | 要指定 | 适用于 |
---|---|---|
| 持久性和 Pod 模板 | 所有 Pod |
| 持久性和 Pod 模板 | 所有 Pod |
| 持久性(忽略 Pod 模板字段) | 集群中的所有分片 |
| Pod Template | 集群中的所有分片 |
| 持久性(忽略 Pod 模板字段) | 集群中的所有分片 |
| Pod Template | 集群中的一个分片 |
| 持久性(忽略 Pod 模板字段) | 特定集群中的一个分片 |
| Pod Template | 特定集群中的一个分片 |
注意
弃用 ShardSpecificPodSpec
覆盖 Pod 模板和持久性设置 clusterSpecList
以下示例说明了如何覆盖 clusterSpecList
中的自定义 Pod 模板和持久性设置。
1 shardPodSpec: # applicable to all shards in all clusters 2 persistence: 3 single: 4 storage: "5G" 5 podTemplate: 6 spec: 7 containers: 8 - name: mongodb-enterprise-database 9 resources: 10 requests: 11 cpu: 0.5 12 memory: 1.0G 13 limits: 14 cpu: 1.0 15 memory: 2.0G 16 shard: 17 clusterSpecList: 18 - clusterName: kind-e2e-cluster-1 19 members: 2 20 # The below statefulset override is applicable only to pods in kind-e2e-cluster-1 21 # Specs will be merged, the "request" field defined above will still be 22 # applied to containers in this cluster. 23 # However, limits will be replaced with below values, because 24 # clusterSpecList.statefulSet.spec.template has a 25 # higher priority than shardPodSpec.podTemplate 26 statefulSet: 27 spec: 28 template: 29 spec: 30 containers: 31 - name: mongodb-enterprise-database 32 resources: 33 limits: 34 cpu: 1.0 35 memory: 2.5G 36 # In clusterSpecList.podSpec, only persistence field must be used, the 37 # podTemplate field is ignored. 38 # In kind-e2e-cluster-1, we replace the persistence settings defined in 39 # shardPodSpec 40 podSpec: 41 persistence: 42 multiple: 43 journal: 44 storage: "6G" 45 data: 46 storage: "7G" 47 logs: 48 storage: "6G" 49 - clusterName: kind-e2e-cluster-2 50 members: 1
要学习;了解更多信息,您可以查看完整文件。
定义自定义 Pod 模板和持久性设置 shardOverrides
以下示例说明了如何在 shardOverrides
中定义自定义 Pod 模板和持久性设置。
1 # In clusterSpecList.podSpec, only persistence field must be used, the 2 # podTemplate field is ignored. 3 # In kind-e2e-cluster-1, we define custom persistence settings 4 podSpec: 5 persistence: 6 multiple: 7 journal: 8 storage: "5G" 9 data: 10 storage: "5G" 11 logs: 12 storage: "5G" 13 - clusterName: kind-e2e-cluster-2 14 members: 1 15 16 shardOverrides: 17 - shardNames: [ "pod-template-shards-1-2" ] 18 # This override will apply to shard of index 2 19 # Statefulset settings defined at this level (shardOverrides.statefulSet) 20 # apply to members of shard 2 in ALL clusters. 21 # This field has higher priority than shard.clusterSpecList.statefulSet, but 22 # lower than shardOverrides.clusterSpecList.statefulSet 23 # It has a merge policy, which means that the limits defined above for the 24 # mongodb-enterprise-database container field still apply to all members in 25 # that shard, except if overridden. 26 statefulSet: 27 spec: 28 template: 29 spec: 30 containers: 31 - name: sidecar-shard-2 32 image: busybox 33 command: [ "sleep" ] 34 args: [ "infinity" ] 35 clusterSpecList: 36 - clusterName: kind-e2e-cluster-1 37 members: 2 38 - clusterName: kind-e2e-cluster-2 39 members: 1 40 # The below statefulset override is applicable only to members of shard 2, in cluster 1 41 # Specs will be merged, the "limits" field defined above will still be applied 42 # to containers in this cluster together with the requests field below. 43 statefulSet: 44 spec: 45 template: 46 spec: 47 containers: 48 - name: mongodb-enterprise-database 49 resources: 50 requests: # We add a requests field in shard 2, cluster 1 51 cpu: 0.5 52 memory: 1.0G 53 54 podSpec: 55 # In shardOverrides.clusterSpecList.podSpec, only persistence field must be 56 # used, the podTemplate field is ignored. 57 persistence: # we assign additional disk resources in shard 2, cluster 1 58 multiple: 59 journal: 60 storage: "6G" 61 data: 62 storage: "6G" 63 logs: 64 storage: "6G"
要学习;了解更多信息,您可以查看完整文件。
从已弃用的 shardSpecificPodSpec
字段迁移
以下示例说明了如何从已弃用的 shardSpecificPodSpec
字段更新到新的 shardOverrides
字段。
1 # This file is an example of how to migrate from the old deprecated 2 # ShardSpecificPodSpec field to the new shardOverrides fields 3 # for single cluster deployments. 4 # The settings specified in shardOverrides are the exact equivalent to the 5 # ones in shardSpecificPodSpec, showing how to replicate them 6 apiVersion: mongodb.com/v1 7 kind: MongoDB 8 metadata: 9 name: shardspecificpodspec-migration 10 namespace: mongodb-test 11 spec: 12 # There are 4 shards in this cluster, but the shardSpecificPodSpec field 13 # doesn't need to have on entry per shard, it can have less 14 shardCount: 4 15 mongodsPerShardCount: 2 16 mongosCount: 1 17 configServerCount: 3 18 topology: SingleCluster 19 type: ShardedCluster 20 version: 8.0.3 21 opsManager: 22 configMapRef: 23 name: my-project 24 credentials: my-credentials 25 persistent: true 26 27 shardPodSpec: 28 # default persistence configuration for all shards in all clusters 29 persistence: 30 single: 31 storage: "5G" 32 shardSpecificPodSpec: # deprecated way of overriding shards (array) 33 - persistence: # shard of index 0 34 single: 35 storage: "6G" 36 # Specify resources settings to enterprise database container in shard 0 37 podTemplate: 38 spec: 39 containers: 40 - name: mongodb-enterprise-database 41 resources: 42 requests: 43 cpu: 0.5 44 memory: 1G 45 limits: 46 cpu: 1.0 47 memory: 2.0G 48 - persistence: # shard of index 1 49 single: 50 storage: "7G" 51 - persistence: # shard of index 2 52 single: 53 storage: "7G" 54 55 # The below shardOverrides replicate the same shards configuration as the one 56 # specified above in shardSpecificPodSpec 57 shardOverrides: 58 - shardNames: [ "shardspecificpodspec-migration-0" ] # overriding shard #0 59 podSpec: 60 persistence: 61 single: 62 storage: "6G" 63 statefulSet: 64 spec: 65 template: 66 spec: 67 containers: 68 - name: mongodb-enterprise-database 69 resources: 70 requests: 71 cpu: 0.5 72 memory: 1G 73 limits: 74 cpu: 1.0 75 memory: 2.0G 76 77 # The ShardSpecificPodSpec field above has the same configuration for shards 78 # 1 and 2. It is possible to specify both shard names in the override and not 79 # duplicate that configuration 80 - shardNames: [ "shardspecificpodspec-migration-1", "shardspecificpodspec-migration-2" ] 81 podSpec: 82 persistence: 83 single: 84 storage: "7G"
要学习;了解更多信息,您可以查看完整文件。
外部访问配置
字段 | 哪些集群 | 哪些分片 |
---|---|---|
| 所有 | 所有 |
| one | 所有 |
| one | 所有 |
Members、MemberConfig、Additional MongodConfig、AgentConfig
ShardedClusterSpec 字段 | 要指定 | 适用于 |
---|---|---|
| 不适用 | 不适用 |
| 适用于 | 适用于 |
| 适用于 | 适用于 |
| 不适用 | 不适用 |
| 适用于 | 适用于 |
配置服务器覆盖
ShardedClusterSpec 字段 | 要指定 | 适用于 |
---|---|---|
| 不适用 | 不适用 |
| 适用于 | 适用于 |
| 不适用 | 不适用 |
| 适用于 | 忽略 |
| 不适用 | 适用于 |
1 configSrv: 2 clusterSpecList: 3 - clusterName: kind-e2e-cluster-1 4 members: 2 5 # The below statefulset override is applicable only to pods in kind-e2e-cluster-1 6 # Specs will be merged, the "request" field defined above will still be applied to containers in this cluster 7 # However, limits will be replaced with below values, because clusterSpecList.statefulSet.spec.template has a 8 # higher priority than configSrvPodSpec.podTemplate 9 statefulSet: 10 spec: 11 template: 12 spec: 13 containers: 14 - name: mongodb-enterprise-database 15 resources: 16 limits: 17 cpu: 1.0 18 memory: 2.5G 19 # In clusterSpecList.podSpec, only persistence field must be used, the podTemplate field is ignored. 20 podSpec: # In kind-e2e-cluster-1, we replace the persistence settings defined in configSrvPodSpec 21 persistence: 22 multiple: 23 journal: 24 storage: "6G" 25 data: 26 storage: "7G" 27 logs: 28 storage: "6G" 29 - clusterName: kind-e2e-cluster-2 30 members: 1 31 # doc-highlight-end: configSrv 32 mongos: 33 clusterSpecList: 34 - clusterName: kind-e2e-cluster-1 35 members: 2 36 - clusterName: kind-e2e-cluster-2 37 members: 1 38 39 shard: 40 clusterSpecList: 41 - clusterName: kind-e2e-cluster-1 42 members: 2 43 - clusterName: kind-e2e-cluster-2 44 members: 1
要学习;了解更多信息,您可以查看完整文件。
Mongos 覆盖
ShardedClusterSpec 字段 | 要指定 | 适用于 |
---|---|---|
| 不适用 | 不适用 |
| 适用于 | 不适用 |
| 不适用 | 适用于 |
| 忽略 | 不适用 |
| 适用于 | 不适用 |