I tried to use golang to insert about 700 billions of documents into one collection on my computer with 2.2 GHz 4 Core Intel Core i7, 16Gb RAM
I perform concurrent method which have 50 gorotines, every gorotines will get one documents from the unbuffered channel and insert one document to the database using InsertOne() func
After let me computer running for a whole night, I wake up and find about 400 millions was inserted on mongodb, while in the mid night, I woke up and find the numbers of documents is about 200 miilions.
I am a new-bie and want to tell you guys about more things i discover and confused.
I think it’s because the number is to big, the number of collection showed in the compass is N/A, so I go to mongosh, use db.collection.countcollections() to get the number, however, it usually takes mins to respond, so that I can not directly know what’s going on in the database.
Is there any better way to insert the millions of data quickly? use other func like buld.insert() or InsertMany()?
why the performance seems to be lower when there’re more documents?
the perfromance of mongodb is slow when it counting millions of documents, is it normal?