Hi,
I have a mongo setup which is sharded and meant for scale.
The services may include multiple config and shard servers (mongod instances), and grow overtime with amount of shards (works on multiple nodes of course).
When I was doing some stress tests with my application I started having some memory issues and saw that mongo instances are taking most of the memory in the system (more than 50%).
Some more info:
My workload consists of many inserts to many different collections, all under the same DB.
The collections that get most of the workload have 3 indexes (regular indexes, not compound).
All the collections are sharded.
I’m using a mongos to balance the workload between my nodes.
A few questions:
Is there a sweetspot for memory usage needed by the mongod instances?
Is there a way to limit ALL the instances together to not exceed 30% of memory consumption? I see that I can put a wiredTigerCacheSizeGB flag per mongod instance but its fixed sized and probably requires restarting the mongod instances and change the values across whenever a mongod instance is added/deleted. I’m looking for a global parameter that all of them will balance to (by the router maybe?).
makes be think you are running multiple mongod on the same hardware machine.
The goal of replica set is for high availability. Running multiple instances of the same replica set within the same machine goes against the goal of high availability.
The goal of shards is to increase capacity and performance. Running multiple shards within the same machine goes against the goal of sharding. Your instances are fighting over the same resources.
Running multiple mongod instances for replica sets or shards withing the same machine is fine for experimentation and to gain experience. Certainly not for
The sweetspot is to have all memory available for a single instance and to run a single instance per physical host.
I know this is an old post, and there may be newer questions about this subject, but I thought I would ask.
I would disagree with this statement: “The sweetspot is to have all memory available for a single instance and to run a single instance per physical host.”
Mongo documentation itself talks about collocated instances, mentioning that it is not recommended for production environments.
So the question stands. We run collocated instances on our development/testing environments. We do build normally 3 replicas each on separate host so we follow the availability model. Now, how do we make sure each instance is using a determined amount of java and cache memory? The way mongodb allocates memory (50%) seems to be that the first instance to start will take over 1/2 of the memory available and the second or 3rd instance will take 50% of whatever is left, and so on.
How can we configure the memory consumption for each instance?