Update all documents in colection only if new data differs from current data

Working with Data
,
1

Hello. Im trying to compare new data to the existing and update all documents in my collection that has diferent values. I using this function but its not working. Can anyone help me. Thank you

Error

Unknown modifier: $or. Expected a valid update modifier or pipeline-style update specified as an array

use('mongodbVSCodePlaygroundDB')

let new_data = [
  { item: 'abc', price: 99, quantity: 999, date: new Date('2014-03-01T08:00:00Z') },
  { item: 'jkl', price: 20, quantity: 1, date: new Date('2014-03-01T09:00:00Z') },
  { item: 'xyz', price: 5, quantity: 10, date: new Date('2014-03-15T09:00:00Z') },
]

// Insert a few documents into the sales collection.

db.getCollection('sales').updateMany( {},
  {
    $or: [
      { item: { $ne: new_data.item } },
      { price: { $ne: new_data.price } },
      { quantity: { $ne: new_data.quantity } },
      { date: { $ne: new_data.date } },
    ],
  },
  {
    $set: { 
      item: new_data.item,
      price: new_data.price,
      quantity: new_data.quantity,
      date: new_data.date,
    },
  },
)
2

Your query has an extra {} in it, the syntax is db.collection.updateMany({filter}, {update}) but you have:

db.collection.updateMany({}. {filter}, {update})

As a start try:

db.getCollection('sales').updateMany(
  {
    $or: [
      { item: { $ne: new_data.item } },
      { price: { $ne: new_data.price } },
      { quantity: { $ne: new_data.quantity } },
      { date: { $ne: new_data.date } },
    ],
  },
  {
    $set: { 
      item: new_data.item,
      price: new_data.price,
      quantity: new_data.quantity,
      date: new_data.date,
    },
  },
)
3

Hello John. I already tryied this way. This works but all sales collections documents values are turning null this way. They are not updating with the new_data documents values.

4

In that case what does new_data look like? Can you output it to the console when running to see the actual value passed?

5

new_data stay the same as i wrote before. And when i write console.log(db.getCollection('sales').find()) in the end of funtion i get:

{
  cursorHasMore: false,
  documents: [
    { _id: {}, item: null, price: null, quantity: null, date: null },
    { _id: {}, item: null, price: null, quantity: null, date: null },
    { _id: {}, item: null, price: null, quantity: null, date: null }
  ]
}
6

Can you show the entire code, I assume you have a loop that goes over new_data? If not and you’re just passing the array as you defined above into the update then it would probably result in nulls.

7

I was think about it. Do i have to loop through each document in my collection? Can you help me elaborate? This is my entire code:

// Select the database to use.
use('mongodbVSCodePlaygroundDB')

let new_data = [
  { item: 'abc', price: 99, quantity: 999, date: new Date('2014-03-01T08:00:00Z') },
  { item: 'jkl', price: 20, quantity: 1, date: new Date('2014-03-01T09:00:00Z') },
  { item: 'xyz', price: 5, quantity: 10, date: new Date('2014-03-15T09:00:00Z') },
]


    db.getCollection('sales').updateMany(
      {
        $or: [
          { item: { $ne: new_data.item } },
          { price: { $ne: new_data.price } },
          { quantity: { $ne: new_data.quantity } },
          { date: { $ne: new_data.date } },
        ],
      },
      {
        $set: {
          item: new_data.item,
          price: new_data.price,
          quantity: new_data.quantity,
          date: new_data.date,
        },
      }
    )

console.log(db.getCollection('sales').find())
8

Apologies, I’ve been away for a few days and didn’t see your response.

Yes, you’ll need to loop over the data, given your code fragment.

If we take one part of your update, say the first condition:

          { item: { $ne: new_data.item } },

new_data is a Javascript array, what this is saying is match where “item” is not equal to the property “item” of the new_data variable.
In this case new_data is an array, and arrays do not have a property called “item” so it’ll be returning undefined, i.e. it’ll not match anything. The same goes for all of the other conditions, so you’ve made a filter criteria that basically matches everything.
Looking at the $set operation, this has the same issue, you’re accessing properties of an array object and not the elements WITHIN the array so they are all null or undefined.

In this scenario, the pipeline operations do not know know to deal with the array.

So one option would be to do something like this:

new_data.forEach(theCurrItem =>{
    db.getCollection('sales').updateMany(
      {
        $or: [
          { item: { $ne: theCurrItem.item } },
          { price: { $ne: theCurrItem.price } },
          { quantity: { $ne: theCurrItem.quantity } },
          { date: { $ne: theCurrItem.date } },
        ],
      },
      {
        $set: {
          item: theCurrItem.item,
          price: theCurrItem.price,
          quantity: theCurrItem.quantity,
          date: theCurrItem.date,
        },
      }
    )
})

You could wrap the above up in a bulk operation to avoid multiple calls to the server:

Hope this makes sense, and again sorry for the delay in getting back to you.

John

9

Hello John how are you? I tryied to implement as you said.

let new_data = [
  { item: 'abc', price: 99, quantity: 999, date: new Date('2014-03-01T08:00:00Z') },
  { item: 'jkl', price: 20, quantity: 1, date: new Date('2014-03-01T09:00:00Z') },
  { item: 'xyz', price: 5, quantity: 10, date: new Date('2014-03-15T09:00:00Z') }
]


new_data.forEach(theCurrItem =>{
    db.getCollection('sales').updateMany(
      {
        $or: [
          { item: { $ne: theCurrItem.item } },
          { price: { $ne: theCurrItem.price } },
          { quantity: { $ne: theCurrItem.quantity } },
          { date: { $ne: theCurrItem.date } },
        ],
      },
      {
        $set: {
          item: theCurrItem.item,
          price: theCurrItem.price,
          quantity: theCurrItem.quantity,
          date: theCurrItem.date,
        },
      },
      {
        upsert: true,
      }
    )
    })

console.log(db.getCollection('sales').find())

And the output was:

{
  cursorHasMore: false,
  documents: [
    {
      _id: ObjectId('65e0a0b524d952bee8941afc'),
      date: 2014-03-15T09:00:00.000Z,
      item: 'xyz',
      price: 5,
      quantity: 10
    }
  ]
}

I was expecting the 3 new_data documents in collection ‘sales’. But its updating only the last document ‘xyz’ item.

10

Hi Renato

let new_data = [
  { item: 'abc', price: 99, quantity: 999, date: new Date('2014-03-01T08:00:00Z') },
  { item: 'jkl', price: 20, quantity: 1, date: new Date('2014-03-01T09:00:00Z') },
  { item: 'xyz', price: 5, quantity: 10, date: new Date('2014-03-15T09:00:00Z') }
]


new_data.forEach(theCurrItem =>{
    db.getCollection('sales').updateMany(
      {
        $or: [
          { item: { $ne: theCurrItem.item } },
          { price: { $ne: theCurrItem.price } },
          { quantity: { $ne: theCurrItem.quantity } },
          { date: { $ne: theCurrItem.date } },
        ],
      },
      {
        $set: {
          item: theCurrItem.item,
          price: theCurrItem.price,
          quantity: theCurrItem.quantity,
          date: theCurrItem.date,
        },
      },
      {
        upsert: true,
      }
    )
    })

console.log(db.getCollection('sales').find())

The problem with the above solutio is in the $or statement, where it checks for all the documents where either of the fields value does not match i.e.

In the loop when the first entry enters as no document in your collection exists, it upserts a new entry with the following -

{item: 'abc', price: 99, quantity: 999, date: new Date('2014-03-01T08:00:00Z') }

When the second entry comes and enters the or statement it returns true at the first check itself as
“jkl” is not equal to “abc” and hence it updates the same document instead of creating a new one, and so in the last you only have the last updated document instead of 3 new documents.

I would suggest that instead of using OR you should use AND operator.

1 Like
11

I tried to substitute the $OR for $AND and the output still is just for the first document. Its not creating the 3 new documents that i expected.

let new_data = [
  { item: 'abc', price: 99, quantity: 999, date: new Date('2014-03-01T08:00:00Z') },
  { item: 'jkl', price: 20, quantity: 1, date: new Date('2014-03-01T09:00:00Z') },
  { item: 'xyz', price: 5, quantity: 10, date: new Date('2014-03-15T09:00:00Z') }
]


new_data.forEach(theCurrItem =>{
    db.getCollection('sales').updateMany(
      {
        $and: [
          { item: { $ne: theCurrItem.item } },
          { price: { $ne: theCurrItem.price } },
          { quantity: { $ne: theCurrItem.quantity } },
          { date: { $ne: theCurrItem.date } },
        ],
      },
      {
        $set: {
          item: theCurrItem.item,
          price: theCurrItem.price,
          quantity: theCurrItem.quantity,
          date: theCurrItem.date,
        },
      },
      {
        upsert: true,
      }
    )
    })

Output

{
  cursorHasMore: false,
  documents: [
    {
      _id: ObjectId('65e0a0b524d952bee8941afc'),
      date: 2014-03-15T09:00:00.000Z,
      item: 'xyz',
      price: 5,
      quantity: 10
    }
  ]
}