MongoDB Freezing

[Srinivas_Dontula]

Good day,
I am new to Mongo DB and able to set up, which is up and running.
So far good.
I have huge data to write into to the mongoDb, I am using console app to write data. The console app is writing data with good speed, however mongo db freezes for few seconds (60-90 seconds) after pushing every 800-1000 records (randomly), this is very frustrating and time consuming as we need to push lots of data.

Please any one advise on this?
Thank you in advance.

[steevej (Steeve Juneau)]

@Srinivas_Dontula

mongo db freezes for few seconds

What makes you conclude that MongoDB is freezing rather than your application?

@Srinivas_Dontula

writing data with good speed

What do you mean by that? Writing to where? Reading from where?

What is your configuration?

Replica set? RAM? CPU? Disk? Everything on same machine or client/server distribution? Network connection? Source of the data? What API are you using?

There is simply not enough information for us to help.

[Srinivas_Dontula]

Hi Steve,

Apologies for not providing the complete info.
Server is: Microsoft Windows 10 Pro With 64 Gig RAM ,16 Cores, using SSD drive with 2 TB.
I am using console app to write documents into Mongo DB, the source files are JSON. Everything is on the same SSD drive.
I am using latest mongo db version (4.4)

during the freeze time i cannot write any documents or search returns null .
I am using .Net drivers to push and pull data from Mongo Db
Please let me know if you want any specific information or screen shots, which I can provide.

Thanks for assisting.

[steevej (Steeve Juneau)]

By API, I mean which insert method? Are you inserting one by one or with bulk write? If your source is JSON files, is there any reason why you are not using https://docs.mongodb.com/database-tools/mongoimport/?

So your mongod and your client are on the same machine and reading and writing from and to the same disk. What is the CPU usage, disk usage when it freeze? Can you do anything else on this machine when your application freeze? Is it the same process that reads and writes from and to MongoDB?

[Srinivas_Dontula]

Hi Steeve,Thanks again.Unfortunately, I cannot use once of import, we receive daily updates, & inserts daily from the customers.Some of them are already in the database and need to be updated (replaced), new ones inserted.
I am first checking if it exists if yes replace (ReplaceOne) else InsertOne.

[steevej (Steeve Juneau)]

@Srinivas_Dontula

I am first checking if it exists if yes replace (ReplaceOne) else InsertOne.

You are missing some fundamental concepts which might be the reason why your application is freezing.

1. For batch processing you definitively should not use the *One() methods. See https://docs.mongodb.com/manual/reference/method/db.collection.bulkWrite/#mongodb-method-db.collection.bulkWrite

2. You can do the logic if-exists-then-replace-else-insert with the concept of upserts. See https://docs.mongodb.com/drivers/node/fundamentals/crud/write-operations/upsert/

[Srinivas_Dontula]

Hi Steeve,
Thank you.
I don’t think it is the case.
Am also trying with bulk writes. Without any updates…
Adding documents to a list and writing to mongoDb using

if (lstToInsert.Count > 0)
await _books.BulkWriteAsync(lstToInsert);
This is also making db to freeze for a while…

I am using default configuration, May be some thing to with configuration ?

Kind Regards