Fastest way to convert 70 million documents

Working with Data
1

I want to convert this schema

{
  total: Number
  leaves: Number
  fakes: Number
  inviterID: String
  fake: Boolean
}

to

{
  invites: Number
  leaves: Number
  fakes: Number
  inviterID: String
  fake: Boolean
}

invites will be calculated using total - leaves. What will be the fastest way to transfer 70 million documents to this layout using that calculation and also deleting the total property?

2

Hi @mental_N_A,

Why not to do a gradually transition for all documents through the application, where the application will have a condition to see if there is a total field in the documen. Then you will calculate the “invites” value and transform the object that use it in new form in app and save to database for future use.

This will not require an unnecessary 70m document migration which might be painful even with multithreading bulk update.

Thanks
Pavel

3 Likes