How to add/remove from Object without overwriting everything

Working with Data
1

I have a collection of documents which have multiple objects within it, and these objects have dates as keys. Like so:

{
   "Data_1":{
      "Thu Oct 21 2021 11:12:00 GMT+0000 (Coordinated Universal Time)":"188245",
      "Sun Oct 24 2021 20:14:00 GMT+0000 (Coordinated Universal Time)":"193033"
      ....
   },
   "Data_2":{
      "Thu Oct 22 2021 17:12:00 GMT+0000 (Coordinated Universal Time)":"abc123",
      "Sun Oct 23 2021 21:19:00 GMT+0000 (Coordinated Universal Time)":"123abc"
      ....
   },
}

These objects can contain hundreds of key/value pairs and I only want to keep data from within the last 365 days.

Currently when I update the document (add new dates and remove old ones) I overwrite the entire object. However as these objects can be reasonably large I believe this is bloating my oplog.

Is there a better way to remove key/value pairs that are older than 365 days and add new dates at the same time without having to overwrite the entire object?

2

Hello @Callum_Boyd, Welcome to the MongoDB community forum,

I am not sure what query you are using for add dates and remove dates, I guess you are passing the whole object of dates in $set or $unset operators,
You have to use . dot notation along with the object name and key name, like this,
For add dates:

{
  "$set": {
    "Data_3.Thu Oct 21 2021 11:12:00 GMT+0000 (Coordinated Universal Time)": "188245",
    "Data_3.Thu Oct 21 2021 11:12:00 GMT+0000 (Coordinated Universal Time)": "123abc"
  }
}

For remove dates:

{
  "$unset": {
    "Data_3.Thu Oct 21 2021 11:12:00 GMT+0000 (Coordinated Universal Time)": "",
    "Data_3.Thu Oct 21 2021 11:12:00 GMT+0000 (Coordinated Universal Time)": ""
  }
}
2 Likes
3

You should not use data values (your dates) as field names. See the attribute pattern for a much better alternative.

You should not store dates as string, it takes more space, it is slower to compare and it takes more bandwidth to transfer.

2 Likes
4

I use the attribute pattern in other collections but had not considered using it here so thank you!

I was also not aware of the downside of storing dates as strings so I appreciate the insight

1 Like
5

Thank you @turivishal, this would certainly make a lot more sense than the way I am currently doing it!

1 Like
6

you may also store each data like this:

{
 "category":"Data_X",
 "date":new Date("2016-05-18T16:00:00Z"),
 "value":"123abc"
}

and this will enable you to write “Time Series Collection” where you can set the automatic deletion of documents after a certain amount of time. please check the following:

7

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.