Hello, there is a production in the IoT sector that prints 5 KB of data for 15000 devices every 30 minutes. My servers are approximately 5 16 core 32 GB ram vm linux. We manage all our applications and db’s through the docker swarm cluster under these 5 VMs. 3 masters 2 workers.
We want to keep this IoT data in mongodb. Can you guide me for the most optimal performance installation? Should it be in Docker swarm under the VM for Mongo, or should I separate it in a separate VM? We are considering indexing the DeviceId and ts fields. What should our Sharding Cluster structure and numbers be? How should I configure CQRS? Currently, I have a system that has 2 shardings via Mongos and 3 replicas in each sharding, but I send the requests to Mongos and it uses a lot of CPU and RAM, and I’m also having performance problems with CQRS.
Hi @yigit_yalnizca and welcome to MongoDB community forums!!
Deploying on an individual VM or on a docker swarm would not make much difference as a docker swarm under the hood converts multiple Docker instances into a single virtual host. Please follow the guidelines for production notes and the hardware and OS configuration to deploy the database successfully.
If you are considering sharding as your deployment, the choice of the shard key would play an important role in this case and hence would suggest you to carefully follow all considerations before you select the shard key.
In saying so, could you help me understand on how does the deployment structure looks like in your case? What are the number of mongos and the shard chunks you are running on the system ?
What is the data size and and what are the operations you are performing?
Could you also help me understand the workload and the performance issues with the current deployment?
Providing this information would help others to give extra context of your use case.
Warm regards
Aasawari