Embed parts of referenced documents and keep them up to date

Working with Data
1

Hello, if i have collection Persons which have name, age, position fields
I also have Houses collection, that have address, floors fields.

But House also needs to have persons list with all persons that live in that particular house.
I understand that i can save list of references persons: [ObjectId] and than populate persons.

but i need to store person name in houses collection persons: [{ref: ObjectId, name: <referenced person name>}]. Data can be updated eventually (asynchrony) but there needs to be guarantees that they will. Or explicitly see which parts have not updated and what why.

Maybe there is something ready.

If not, my best idea currently is as fallows: use changeStreams with cursor (so no changes have been missed out on) + each change initiates agenda job if needed to replicate data (so there are explicit success/failure for each particular “replication”).

Im not entirely happy with this solution as i think it could get slow with large amount of data. Maybe there are other caveats to such approach. Maybe if there isn’t some out of the box solution, maybe there are some other way how to approach this?

Other way was to on save, trigger save for other places as well (as agenda), but ideally things would update correctly even if changed directly in db, or from other app. Change stream would handle those cases as i understand.

2

One way to achieve what you want is to use MongoDB’s embedded documents instead of references. You can create a subdocument for each person in the Houses collection, which contains both the ObjectId reference and the person’s name.

For example, your Houses collection might look like this:

{
  "_id": ObjectId("..."),
  "address": "123 Main St",
  "floors": 2,
  "persons": [
    {
      "ref": ObjectId("..."),
      "name": "John"
    },
    {
      "ref": ObjectId("..."),
      "name": "Jane"
    }
  ]
}

This way, you can store the person’s name along with their reference, and you won’t have to perform additional queries to retrieve the names.

To update the person names, you can use a multi-update operation to update all occurrences of a person’s ObjectId in the Houses collection. For example:

db.Houses.updateMany(
  { "persons.ref": ObjectId("...") },
  { $set: { "persons.$[].name": "New Name" } }
);

This will update all occurrences of the person with the given ObjectId in the Houses collection.

To ensure that updates to the Persons collection are reflected in the Houses collection, you can use change streams as you suggested. When a person document is updated, you can use a change stream to find all Houses documents that reference that person, and update the person name in those documents.

Using a cursor with change streams should not be slow, as long as you handle each change efficiently. You can use an asynchronous job queue like Agenda to handle updates, which will allow you to process changes in the background without affecting the performance of your application.