Indexing while inserting

Question 1: I have java threads which are working on mongo DB . when Thread A is inserting records on Collection A and after a while Thread B creating indexes on Collection B, mongo is not processing Thread A insertion commands. How can we make sure it will work parlerly .

Question 2: I have data ready in relational data , just want to synch that to mongo. in this case Indexing after insertion ? or insertion after indexing ? which one is better .

Most likely your setup is struggling from insufficient hardware. Or you are using an old mongod, but even then I do not think that creating index on collection B will affect insert on collection A. So insufficient hardware, thus excessive context switch.

None is better. The total amount of work is similar. It is a tradeoff, it depends on your use-case. Creating the index before slows down insertion but your data is ready for quick queries when all insertions are done. Creating the index after speeds up insertion, but you have to wait after index creation before you have speedy queries.

1 Like

Thanks Steve for your reply . Still I have doubt in Question 1:
Sorry Steve, I do not think that I am suffering from hardware problem , because we are getting this problem in production loud. Mongo is not issuing commands Collection B while indexing is going on for Collection A . I can similar kind of behavior from this page,
Index Build Operations on a Populated Collection — MongoDB Manual , can you please explain me below statement from this page .

Any operation that requires a read or write lock on all databases (e.g. listDatabases ) will wait for the foreground index build to complete.

Background indexing operations run in the background so that other database operations can run while creating the index. However, the [ mongo ]shell session or connection where you are creating the index will block until the index build is complete. To continue issuing commands to the database, open another connection or [ mongo ] instance.

Hi @Puvvada_Divya welcome to the community!

Mongo is not issuing commands Collection B while indexing is going on for Collection A

The behaviour you’re seeing was actually mentioned in the page you linked, specifically: By default, creating an index on a populated collection blocks all other operations on a database. So this is in line with what you experienced.

However I noticed that you’re linking to the documentation page for the 4.0 series, which is not supported anymore since April 2022. Are you using MongoDB 4.0 series by any chance?

This behaviour was improved in MongoDB 4.2: Starting in MongoDB 4.2, index builds use an optimized build process that holds an exclusive lock on the collection at the beginning and end of the index build. The rest of the build process yields to interleaving read and write operations.

It sounds like you’re trying to create a new MongoDB deployment. If this is a fresh deployment, I would suggest you use the latest MongoDB version (5.0.8 currently) instead of using an unsupported version. You will also not see this issue if you’re using the latest version.

Best regards
Kevin

3 Likes