Who is responsible for balancer process mongos or primary node?

@0:50 in the video and consistent with the documentation he said that mongos USED to be responsible for the balancer but in new version the primary node of CSRS is now responsible for this process. Then @ about 1:44 he seemed so say something different. “typically mongos handles a chunk split,…”. Does this mean that mongos CAN still perform splits but under most circumstances it does not? If I am reading this correctly what are those special circumstances. OR is this as I suspect my own bleary-eyed confusion after to much mindtickling?

Hi @Dwight_Ford1,

mongos processes are no longer responsible for chunk splits or migrations in modern versions of MongoDB – they are now focused on routing commands and returning results. Shard primaries manage chunk splits; chunk migration (aka balancing) is coordinated via the config server primary.

Chunk splits are local to a shard and do not require coordination with other shards as they are just a metadata change (splitting a single range of shard key values into 2 or more chunk ranges on the same shard). Typically chunk splits happen automatically on the shard primary as data is inserted and updated via mongos.

Chunk balancing need to be coordinated between multiple shards. The balancer process runs on the config server primary in modern MongoDB deployments. If the balancer process chooses a chunk range for migration that needs to be split, the split will be coordinated via the shard primary currently owning the chunk. In most cases chunks should already be appropriately sized, so the balancer may not have to request any splits.

A deeper dive

The READMEs in the server source code on GitHub have a detailed overview including code links. I’ve include some relevant quotes from Sharding Internals (mongo/src/mongo/db/s/README.md), but there’s a lot more detail if you’re curious :slight_smile:

Shards have several internal services including:

  • ChunkManager: Every time an update or insert gets routed to a chunk, the server tracks the bytes written to the chunk in memory through the collection’s ChunkManager. The ChunkManager has a ChunkInfo object for each of the collection’s entries in the local config.chunks. Through the ChunkManager, the server retrieves the chunk’s ChunkInfo and uses its ChunkWritesTracker to increment the estimated chunk size.

  • ChunkSplitter: The ChunkSplitter is a replica set primary-only service that manages the process of auto-splitting chunks. The ChunkSplitter runs auto-split tasks asynchronously - thus, distinct chunks can undergo an auto-split concurrently.

Auto-splitting

When the mongos routes an update or insert to a chunk, the chunk may grow beyond the configured chunk size (specified by the server parameter maxChunkSizeBytes) and trigger an auto-split, which partitions the oversized chunk into smaller chunks. The shard that houses the chunk is responsible for:

  • determining if the chunk should be auto-split
  • selecting the split points
  • committing the split points to the config server
  • refreshing the routing table cache
  • updating in memory chunk size estimates

Auto-balancing

The balancer is a background process that monitors the chunk distribution in a cluster. It is enabled by default and can be turned off for the entire cluster or per-collection at any time.

The balancer process runs in a separate thread on the config server primary. It runs continuously, but in “rounds” with a 10 second delay between each round. During a round, the balancer uses the current chunk distribution and zone information for the cluster to decide if any chunk migrations or chunk splits are necessary.

Regards,
Stennie

2 Likes

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.