local ssd disk.basically everything is local and the one replica set is for supporting transactions
4 cores 3.4 GHz
it is not shared, not a VM, normal Ubuntu desktop IMac computer
create collection is a single line simple function call in Java driver , no strings attatched
also the insertMany is a simple function call
there are no other instructions in between that might cause the delay
cpu is between 50 to 80 percent and Ram is 10GB.
and i dont have any extra index that might effect insertions.
the weird thing is creating collections.it seems it is a very costly task, am i right?
what is a normal expectation for creating a collection in a system described as above?
actually when i run sequentially all create collection and insertions run less the 0.01 sec.
the blog and the youtube video that descrives 6 anti-pattern of schema design was a great help.i think i am misusing Mongo.because based on my design, i am creating a new collection for each Transaction and then immediately i drop it.but i have more than 20000 collections active at the same time due to load on the server.so as i understood from the youtube video, the WiredTiger creates 2 files for each collection, one for data and one for each index and by default the Wiredtiger opens all of the files at the startup as much as it hits it’s Cache which is half of the Ram.and as recommended, i have to decrease the collections to under 10000 in a single replica set.so i have to change my design.i monitored the disk io and it was almost 90 percent all the time used by the mongo process.so i think you have helped me to pin point the problem.
let me change my design to include all of them in a single collection and finding them using indexes.i’ll let you know about the results.i’ll try not to step to the other pitfalls explained in the video, like unused indexes or bloated documents or massive arrays.
thank you man, you helped me alot.i’ll post the new stats to help others.
sorry for late response.you were absolutely right.that video helped a lot.we should not have more than 10000 collection per replica set.so the moment i changed my schema design to single collection with a wildcard index, it was like a miracle.everything is super sonic now.i had careful abstractions and anti corruption layers over my infrastructure which make it possible for me to switch from mongo to sth else in an hour for and enterprise application with more than 10 bounded contexts and allowed me to change my design from thousands of collection, into a single or two collection.but if you hardcode your schema to your code, you will have a hard time modifying your schema design.thank you anyway, you helped me alot