Docs 菜单
Docs 主页
/
Enterprise Kubernetes Operator
/

多集群分片集群

在此页面上

  • 限制
  • 先决条件
  • 分片集群部署示例
  • 分片覆盖
  • 自定义 Persistence 和 Statefulset 设置
  • 外部访问配置
  • Members、MemberConfig、Additional MongodConfig、AgentConfig
  • 配置服务器覆盖
  • Mongos 覆盖

您可以将MongoDB分片集群分布在多个Kubernetes集群上。通过多集群功能,您可以:

  • 通过将部署分布在多个Kubernetes集群(每个集群位于不同的地理地区)中,提高部署的韧性。

  • 通过在不同Kubernetes集群中部署指定分片的主节点 (primary node in the replica set)节点来配置地理分片的部署,这些集群距离依赖于该数据的应用程序或客户端更近,从而减少延迟。

  • 调整部署以提高性能。示例,您可以为不同Kubernetes集群中的所有分片或指定分片部署只读分析节点,也可以自定义资源分配。

此外,您可以在不同级别配置分片、 mongos和配置服务器详细信息。这意味着您可以为分片、配置服务器和mongos定义顶级默认配置,并为每个Kubernetes集群独立进行自定义。此外,还可以自定义单个分片以满足您的特定需求。

重要

多集群分片分片集群功能使得在多个地理区域的多个Kubernetes集群上部署MongoDB资源成为可能;但是,这样做可能会增加复制操作的延迟。

要学习;了解有关多集群分片集群拓扑结构的特定配置选项的更多信息,请参阅 分片集群参考。

  • Hashicorp Vault支持不适用于Kubernetes密钥注入。

  • Kubernetes Operator 不会自动将资源从出现故障的Kubernetes集群转移到正常集群。如果成员集群发生故障,您必须通过更新部署到中央集群的 CRD资源,手动在剩余健康Kubernetes集群之间重新分配资源。

  • 如果成员集群发生故障,必须首先手动扩展故障集群缩减至零,以删除运行状况不佳的进程。然后,您可以通过更新部署到中央集群的CRD资源,手动在剩余健康Kubernetes集群之间重新分配资源。请参阅灾难恢复以学习;了解更多信息。

当应用于中央集群时,以下示例部署了一个包含 3 分片的分片MongoDB 集群,配置如下:

  • 每个分片都有分布式在所有Kubernetes集群的节点(每个集群1 个节点)。

  • 默认下,每个分片的副本集:

    • 在 3 个集群中总共部署了 3 个有投票权的成员(每个集群中有 1 个节点)

    • 主分片最好位于 kind-e2e-cluster-1 中。

  • 通过分片sc-0shardOverride,我们将默认值(在 spec.shard 中指定)更改为以下内容:

    • 总共 5 个成员

    • 2 成员位于 kind-e2e-cluster-1

    • 2 成员位于 kind-e2e-cluster-2

    • 1 成员 kind-e2e-cluster-3

    • 如果可能,主分片最好位于 kind-e2e-cluster-2

  • 为所有集群中的所有分片自定义默认存储设置,如 spec.shardPodSpec

  • 具有 3 个成员的配置服务器副本集,每个集群具有 1 个成员

  • 总共部署 3 个mongos实例,每个集群有 1 个实例

默认配置由三个分片组成,每个分片在三个集群上各有一个成员,每个分片总共有三个成员。但在覆盖中,我们将分片sc-0 更改为具有五个成员,其中两个在集群1 上,两个在集群2 上,并且集群3 仍然具有默认的一个分片。

注意

仅支持单向扩展。

当您扩展分片、配置服务器或mongos副本时,您只能向上或向下扩展资源,以确保扩展的正确性。此规则适用于单个Kubernetes集群部署和分布式在多个Kubernetes集群的部署。此外,在多集群部署中,单向扩展规则适用于所有资源类型。示例,您不能在删除(扩展)配置服务器或mongos 的同时向分片添加更多节点(向上扩展)。在不更改成员总数的情况下,您无法再将一个节点从一个集群“移动”到另一个集群。相反,您必须首先执行纵向扩展操作,然后执行单独的扩展更改。

对于分片sc-0,此示例配置还会(使用 shardOverrides)将主节点 (primary node in the replica set)转移到集群2,这可以减少在集群2 所在地区中操作的用户的延迟。这样,您仍然具有跨集群(及其区域)的韧性,但如果分片0 中的数据是与部署集群2 的用户最相关的数据,则他们将体验到更低的延迟。

1apiVersion: mongodb.com/v1
2kind: MongoDB
3metadata:
4 name: sc
5spec:
6 topology: MultiCluster
7 type: ShardedCluster
8 # this deployment will have 3 shards
9 shardCount: 3
10 # you cannot specify mongodsPerShardCount, configServerCount and mongosCount
11 # in MultiCluster topology
12 version: 8.0.3
13 opsManager:
14 configMapRef:
15 name: my-project
16 credentials: my-credentials
17 persistent: true
18
19 shardPodSpec: # applies to all shards on all clusters
20 persistence:
21 single:
22 # all pods for all shards on all clusters will use that storage size in their
23 # PersistentVolumeClaim unless overridden in spec.shard.clusterSpecList or
24 # spec.shardOverrides.
25 storage: 10G
26
27 configSrvPodSpec: # applies to all config server nodes in all clusters
28 persistence:
29 multiple:
30 data:
31 storage: 2G
32 journal:
33 storage: 1G
34 logs:
35 storage: 1G
36
37 # consider this section as a default configuration for ALL shards
38 shard:
39 clusterSpecList:
40 - clusterName: kind-e2e-cluster-1
41 # each shard will have only one mongod process deployed in this cluster
42 members: 1
43 memberConfig:
44 - votes: 1
45 priority: "20" # we increase the priority to have primary in this cluster
46 - clusterName: kind-e2e-cluster-2
47 # one member in this cluster, no votes and priority defined means it'll get
48 # the default values votes=1, priority="1"
49 members: 1
50 - clusterName: kind-e2e-cluster-3
51 members: 1 # one member in this cluster
52
53 shardOverrides: # here you specify customizations for specific shards
54 # here you specify to which shard names the following configuration will
55 # apply
56 - shardNames:
57 - sc-0
58 clusterSpecList:
59 - clusterName: kind-e2e-cluster-1
60 # all fields here are optional
61 # shard "sc-0" will have two members instead of one, which was defined as the
62 # default for all shards in spec.shard.clusterSpecList[0].members
63 members: 2
64 memberConfig:
65 - votes: 1
66 # shard "sc-0" should not have primary in this cluster like every other shard
67 priority: "1"
68 - votes: 1
69 priority: "1"
70 - clusterName: kind-e2e-cluster-2
71 members: 2 # shard "sc-0" will have two members instead of one
72 memberConfig:
73 - votes: 1
74 # both processes of shard "sc-0" in this cluster will have the same
75 # likelihood to become a primary member
76 priority: "20"
77 - votes: 1
78 priority: "20"
79 # We need to specify the list of all clusters on which this shard will be
80 # deployed.
81 - clusterName: kind-e2e-cluster-3
82 # If the clusterName element is omitted here, it will be considered as an
83 # override for this shard, so that the operator shouldn't deploy any member
84 # to it.
85 # No fields are mandatory in here, though. In case a field is not set, it's
86 # not overridden and the default value is taken from a top level spec.shard
87 # settings.
88
89 configSrv:
90 # the same configuration fields are available as in
91 # spec.shard.clusterSpecList.
92 clusterSpecList:
93 - clusterName: kind-e2e-cluster-1
94 members: 1
95 - clusterName: kind-e2e-cluster-2
96 members: 1
97 - clusterName: kind-e2e-cluster-3
98 members: 1
99
100 mongos:
101 # the same configuration fields are available as in
102 # spec.shard.clusterSpecList apart from storage and replica-set related
103 # fields.
104 clusterSpecList:
105 - clusterName: kind-e2e-cluster-1
106 members: 1
107 - clusterName: kind-e2e-cluster-2
108 members: 1
109 - clusterName: kind-e2e-cluster-3
110 members: 1

要将分片集群部署到多个Kubernetes集群,您应用分片集群配置(MongoDB自定义资源yaml)应用于操作符集群,即部署MongoDB Operator实例的Kubernetes集群。此配置的 spec.shard 定义被视为部署的基本分片定义。

如果要在特定Kubernetes集群上自定义特定分片,可以使用分片覆盖来更新给定分片的基本分片定义。

下表列出了您可以定义以更新或扩展基本分片定义的字段。字段按优先顺序列出。给定表中的最顶层字段表示优先级最低的设置,最底层字段(如果已定义)将覆盖所有其他字段(例如) 。

此外,为每种字段类型表示的覆盖策略描述了该特定字段是否被新定义的值覆盖,或者定义该字段的完整对象是否被覆盖。如果覆盖了整个对象,则在基本分片定义中定义且未在覆盖定义中显式定义的任何字段都将被删除。merge 值表示更新单个字段,replace 值表示更新单个字段。完整的父对象被覆盖。

ShardedClusterSpec 字段
要指定
适用于

spec.shardPodSpec

持久性和 Pod 模板

所有 Pod

spec.shardSpecificPodSpec (已弃用)

持久性和 Pod 模板

所有 Pod

spec.shard.clusterSpecList.podSpec

持久性(忽略 Pod 模板字段)

集群中的所有分片

spec.shard.clusterSpecList.statefulSet

Pod Template

集群中的所有分片

spec.shardOverrides.podSpec

持久性(忽略 Pod 模板字段)

集群中的所有分片

spec.shardOverrides.statefulSet

Pod Template

集群中的一个分片

spec.shardOverrides.clusterSpecList.podSpec

持久性(忽略 Pod 模板字段)

特定集群中的一个分片

spec.shardOverrides.clusterSpecList.statefulSet

Pod Template

特定集群中的一个分片

注意

弃用 ShardSpecificPodSpec

ShardSpecificPodSpec字段已弃用,但仍受支持。它以前用于为单个集群分片集群指定每个分片的持久性和 Pod 模板参数。现在它已被弃用,您应该从 迁移到ShardOverrides.PodSpecShardOverrides.StatefulSetConfiguration 。有关从单集群部署的 shardSpecificPodSpecshardOverrides 进行更新的指导,请参阅提供的示例YAML文件。

以下示例说明了如何覆盖 clusterSpecList 中的自定义 Pod 模板和持久性设置。

1 shardPodSpec: # applicable to all shards in all clusters
2 persistence:
3 single:
4 storage: "5G"
5 podTemplate:
6 spec:
7 containers:
8 - name: mongodb-enterprise-database
9 resources:
10 requests:
11 cpu: 0.5
12 memory: 1.0G
13 limits:
14 cpu: 1.0
15 memory: 2.0G
16 shard:
17 clusterSpecList:
18 - clusterName: kind-e2e-cluster-1
19 members: 2
20 # The below statefulset override is applicable only to pods in kind-e2e-cluster-1
21 # Specs will be merged, the "request" field defined above will still be
22 # applied to containers in this cluster.
23 # However, limits will be replaced with below values, because
24 # clusterSpecList.statefulSet.spec.template has a
25 # higher priority than shardPodSpec.podTemplate
26 statefulSet:
27 spec:
28 template:
29 spec:
30 containers:
31 - name: mongodb-enterprise-database
32 resources:
33 limits:
34 cpu: 1.0
35 memory: 2.5G
36 # In clusterSpecList.podSpec, only persistence field must be used, the
37 # podTemplate field is ignored.
38 # In kind-e2e-cluster-1, we replace the persistence settings defined in
39 # shardPodSpec
40 podSpec:
41 persistence:
42 multiple:
43 journal:
44 storage: "6G"
45 data:
46 storage: "7G"
47 logs:
48 storage: "6G"
49 - clusterName: kind-e2e-cluster-2
50 members: 1

要学习;了解更多信息,您可以查看完整文件。

以下示例说明了如何在 shardOverrides 中定义自定义 Pod 模板和持久性设置。

1 # In clusterSpecList.podSpec, only persistence field must be used, the
2 # podTemplate field is ignored.
3 # In kind-e2e-cluster-1, we define custom persistence settings
4 podSpec:
5 persistence:
6 multiple:
7 journal:
8 storage: "5G"
9 data:
10 storage: "5G"
11 logs:
12 storage: "5G"
13 - clusterName: kind-e2e-cluster-2
14 members: 1
15
16 shardOverrides:
17 - shardNames: [ "pod-template-shards-1-2" ]
18 # This override will apply to shard of index 2
19 # Statefulset settings defined at this level (shardOverrides.statefulSet)
20 # apply to members of shard 2 in ALL clusters.
21 # This field has higher priority than shard.clusterSpecList.statefulSet, but
22 # lower than shardOverrides.clusterSpecList.statefulSet
23 # It has a merge policy, which means that the limits defined above for the
24 # mongodb-enterprise-database container field still apply to all members in
25 # that shard, except if overridden.
26 statefulSet:
27 spec:
28 template:
29 spec:
30 containers:
31 - name: sidecar-shard-2
32 image: busybox
33 command: [ "sleep" ]
34 args: [ "infinity" ]
35 clusterSpecList:
36 - clusterName: kind-e2e-cluster-1
37 members: 2
38 - clusterName: kind-e2e-cluster-2
39 members: 1
40 # The below statefulset override is applicable only to members of shard 2, in cluster 1
41 # Specs will be merged, the "limits" field defined above will still be applied
42 # to containers in this cluster together with the requests field below.
43 statefulSet:
44 spec:
45 template:
46 spec:
47 containers:
48 - name: mongodb-enterprise-database
49 resources:
50 requests: # We add a requests field in shard 2, cluster 1
51 cpu: 0.5
52 memory: 1.0G
53
54 podSpec:
55 # In shardOverrides.clusterSpecList.podSpec, only persistence field must be
56 # used, the podTemplate field is ignored.
57 persistence: # we assign additional disk resources in shard 2, cluster 1
58 multiple:
59 journal:
60 storage: "6G"
61 data:
62 storage: "6G"
63 logs:
64 storage: "6G"

要学习;了解更多信息,您可以查看完整文件。

以下示例说明了如何从已弃用的 shardSpecificPodSpec字段更新到新的 shardOverrides字段。

1# This file is an example of how to migrate from the old deprecated
2# ShardSpecificPodSpec field to the new shardOverrides fields
3# for single cluster deployments.
4# The settings specified in shardOverrides are the exact equivalent to the
5# ones in shardSpecificPodSpec, showing how to replicate them
6apiVersion: mongodb.com/v1
7kind: MongoDB
8metadata:
9 name: shardspecificpodspec-migration
10 namespace: mongodb-test
11spec:
12 # There are 4 shards in this cluster, but the shardSpecificPodSpec field
13 # doesn't need to have on entry per shard, it can have less
14 shardCount: 4
15 mongodsPerShardCount: 2
16 mongosCount: 1
17 configServerCount: 3
18 topology: SingleCluster
19 type: ShardedCluster
20 version: 8.0.3
21 opsManager:
22 configMapRef:
23 name: my-project
24 credentials: my-credentials
25 persistent: true
26
27 shardPodSpec:
28 # default persistence configuration for all shards in all clusters
29 persistence:
30 single:
31 storage: "5G"
32 shardSpecificPodSpec: # deprecated way of overriding shards (array)
33 - persistence: # shard of index 0
34 single:
35 storage: "6G"
36 # Specify resources settings to enterprise database container in shard 0
37 podTemplate:
38 spec:
39 containers:
40 - name: mongodb-enterprise-database
41 resources:
42 requests:
43 cpu: 0.5
44 memory: 1G
45 limits:
46 cpu: 1.0
47 memory: 2.0G
48 - persistence: # shard of index 1
49 single:
50 storage: "7G"
51 - persistence: # shard of index 2
52 single:
53 storage: "7G"
54
55 # The below shardOverrides replicate the same shards configuration as the one
56 # specified above in shardSpecificPodSpec
57 shardOverrides:
58 - shardNames: [ "shardspecificpodspec-migration-0" ] # overriding shard #0
59 podSpec:
60 persistence:
61 single:
62 storage: "6G"
63 statefulSet:
64 spec:
65 template:
66 spec:
67 containers:
68 - name: mongodb-enterprise-database
69 resources:
70 requests:
71 cpu: 0.5
72 memory: 1G
73 limits:
74 cpu: 1.0
75 memory: 2.0G
76
77 # The ShardSpecificPodSpec field above has the same configuration for shards
78 # 1 and 2. It is possible to specify both shard names in the override and not
79 # duplicate that configuration
80 - shardNames: [ "shardspecificpodspec-migration-1", "shardspecificpodspec-migration-2" ]
81 podSpec:
82 persistence:
83 single:
84 storage: "7G"

要学习;了解更多信息,您可以查看完整文件。

字段
哪些集群
哪些分片

spec.externalAccess

所有

所有

spec.shard.externalAccess

one

所有

spec.shard.clusterSpecList.externalAccess

one

所有

ShardedClusterSpec 字段
要指定
适用于

spec.shard

不适用

不适用

spec.shard.clusterSpecList

适用于

适用于

spec.shardOverrides (仅限单个集群)

适用于

适用于

spec.sharOverrides

不适用

不适用

spec.shardOverrides.clusterSpecList

适用于

适用于

ShardedClusterSpec 字段
要指定
适用于

spec.configSrv

不适用

不适用

spec.configSrv

适用于

适用于

spec.configSrv.clusterSpecList

不适用

不适用

spec.configSrv.clusterSpecList.podSpec

适用于

忽略

spec.configSrv.clusterSpecList.statefulSet

不适用

适用于

1 configSrv:
2 clusterSpecList:
3 - clusterName: kind-e2e-cluster-1
4 members: 2
5 # The below statefulset override is applicable only to pods in kind-e2e-cluster-1
6 # Specs will be merged, the "request" field defined above will still be applied to containers in this cluster
7 # However, limits will be replaced with below values, because clusterSpecList.statefulSet.spec.template has a
8 # higher priority than configSrvPodSpec.podTemplate
9 statefulSet:
10 spec:
11 template:
12 spec:
13 containers:
14 - name: mongodb-enterprise-database
15 resources:
16 limits:
17 cpu: 1.0
18 memory: 2.5G
19 # In clusterSpecList.podSpec, only persistence field must be used, the podTemplate field is ignored.
20 podSpec: # In kind-e2e-cluster-1, we replace the persistence settings defined in configSrvPodSpec
21 persistence:
22 multiple:
23 journal:
24 storage: "6G"
25 data:
26 storage: "7G"
27 logs:
28 storage: "6G"
29 - clusterName: kind-e2e-cluster-2
30 members: 1
31 # doc-highlight-end: configSrv
32 mongos:
33 clusterSpecList:
34 - clusterName: kind-e2e-cluster-1
35 members: 2
36 - clusterName: kind-e2e-cluster-2
37 members: 1
38
39 shard:
40 clusterSpecList:
41 - clusterName: kind-e2e-cluster-1
42 members: 2
43 - clusterName: kind-e2e-cluster-2
44 members: 1

要学习;了解更多信息,您可以查看完整文件。

ShardedClusterSpec 字段
要指定
适用于

ConfigSrvSpec

不适用

不适用

ConfigSrvPodSpec

适用于

不适用

ConfigSrvSpec.ClusterSpecList

不适用

适用于

ConfigSrvSpec.ClusterSpecList.PodSpec

忽略

不适用

ConfigSrvSpec.ClusterSpecList.StatefulSetConfiguration

适用于

不适用

后退

从外部Kubernetes连接