Best way to do a batch upsert?

Hi, what is the best way to do a batch upsert using Mongoose or Nodejs driver?
Assuming this data:

db.users.insert([
   { _id: "1", name: "koko", age: 15 },
   { _id: "2", name: "momo", age: 32 },
   { _id: "3", name: "charles", age: 73 }
]);

We want to do a batch upsert that will result creating 2 more documents and updating some of the existing ones:
Input data for batch upsert:

[
  { _id: "1", name: "koko sho", age: 17 }, // updates name **and** age
 { _id: "2", name: "momo", age: 39 }, // updates only age
 { _id: "3", name: "charles", age: 73 }, // same
 { _id: "4", name: "dian", age: 43 }, // new
 { _id: "5", name: "joe", age: 33 } // new
]

which will result:

[
 { _id: "1", name: "koko sho", age: 17 } // age and name were updated because the upsert had { _id: "1", age: 17 },
 { _id: "2", name: "momo", age: 39 }, // only age was updated
 { _id: "3", name: "charles", age: 73 }, // same
 { _id: "4", name: "dian", age: 43 }, // new
 { _id: "5", name: "joe", age: 33 } // new
]

How to achieve it with a 1 upsert operation?

hi @Benny_Kachanovsky1

I think the most straightforward way to do this is to use mongoose’s bulkWrite operator, specifying multiple updateOne operations to be executed in bulk. To update or insert a new document, specify the {upsert: true} option.

For example:

    let newUsers = [
      { _id: "1", name: "koko sho", age: 17 }, // updates name **and** age
      { _id: "2", name: "momo", age: 39 }, // updates only age
      { _id: "3", name: "charles", age: 73 }, // same
      { _id: "4", name: "dian", age: 43 }, // new
      { _id: "5", name: "joe", age: 33 } // new
    ]

    for (let i in newUsers) {
      console.log(newUsers[i])
      newUsers[i] = {
        updateOne: {
          filter: {_id: newUsers[i]._id},
          update: newUsers[i],
          upsert: true
        }
      }
    }

    const res = await User.bulkWrite(newUsers)

In the for loop in the example above, all I did was wrap the documents into a format that bulkWrite requires.

MongoDB would not modify a document if the update operation will not change anything, so the document with _id: "3" in your example will not be touched.

Best regards
Kevin

1 Like