Mongo schema for many to few relationship - users following few products - approach?

There have been many similar questions but they are all about followers/followees where each other is following the other (bot sides could potentially grow very large)

my case: I have users, each user can follow products. Each user has a watchlist (summary of all watched items). I want to be able to send notification to all users who follow a spec. product when something about the product has changed. So I need to query all followers of a specific product (there will be many products but they will never grow as big as the document size limit would be reached)

first approach

pro: only one lookup to get followerIds (lookup of collection followed)

con: 16mb per document limit would be potentially reached because followeeIds in followed could grow very big

1. collection users: 
  - userid 
  - username 
  - userpassword 
  - other info user  
2. collection following: 
  - userid 
  - [array of followingIds] //productIds 
3. collection followed (watchlist): 
  - productId 
  - [array of followeeIds] 

second approach

pro: 16mb per document limit would not be reached

con: would be 2 queries as I understand -have to search through all users + followeeIds to see who is follower

1. collection users: 
  - userid - username 
  - userpassword 
  - other info user 
  - followeeIds: [] //ids of products user follows 
2. collection followed (watchlist) 
  - userId 
  - reference to product or embedded product 

3rd approach:

I would reference users in my followee collection. Im not sure if this would lead to less queries/better performance


1. collection users: 
  - userid - username 
  - userpassword 
  - other user specific info user  
2. collection following: 
  - userid 
  - [array of followingIds] //productIds (no reference as it would never grow as large as userIds) 
3. collection followed: 
  - productId 
  - followee: {
    type: Schema.Types.ObjectId,         
    ref: 'user'      
   } //here instead of having array of userIds, I reference user

4th approach: here I would not use arrays so lets say user 1 follows 5 products, there will be 5 documents belonging to collection Follow.

1. collection users: 
  - userid - username 
  - userpassword 
  - other info user 
2. collection follow 
  - userid (the follower)
  -productId (followee)
3. collection Watchlist (this is to show users watchlist, not for sending notifications to all followers of product xyz
  - userId
  -embedded or reference to product

Hi @Anna_N_A,

I think you haven’t considered a hibread solution where you would embedded a portion of users following products in its main document and have any overflow products in an “overflow” document.

collection users: 
  - userid - username 
 - type : "main"/"overflow"
  - userpassword 
  - other info user 
  - followeeIds: [{productId, name ...}]
 - hasOverflow : true/false
collection products:
- any additional products info

So whenever your wish list is growing lets say beyond 200 products you open an overflow document in the same collection. You index the user id and also index the product id. When you search for a product id to notify you will get the user information from any document as you will duplicate that into each overflow document.

Or if you search for the user entire list you will get all of its documents based on the user index. All done with one query no need for lookups

This is called outlier pattern:

Thanks
Pavel

3 Likes

Thanks a lot for your answer. Based on the article you shared, the outlier pattern is used more in exceptional cases, where certain documents could become too large (the example of the harry potter book for example). Should this pattern also be applied, when its likely that many (not most but neither only a few) documents will be affected and outlier pattern will be applied? Also, what number approx. would you recommend to start applying that pattern?

Hi @Anna_N_A ,

Having large arrays you use for query or write operations is a place to be cautious about.

Therefore if your arrays cross the houndreds i would split them into outlier pattern.

Its also good for ui pagination if you think about it.

My place to start for small to medium (few kb ) elements is around 500 (max 1000) per document/bucket…

Thanks
Pavel

1 Like

Just one last question. Would you also apply the outlier pattern to global notifications, where notifications are send to all users? something like this:

collection globalNotifications (notifications send to all users)
to:  [{   //ObjectId's indexed
        type: mongoose.Schema.Types.ObjectId,
        ref: 'User'
    }],
    readAt: {
        type: Date
    },
   hasOverflow : true
...

or would you in this case because all users will be affected still choose this:

collection globalNotifications (notifications send to all users)
to: {   //ObjectId indexed
        type: mongoose.Schema.Types.ObjectId,
        ref: 'User'
    },
    readAt: {
        type: Date
    },
...

Please share the nature of a notification?

Is it added when something change or a user login?

I have 2 types of notifications.

  • global notifications: send to all users without any ‘trigger’ by any user or other action (just broadcast news)
  • notification triggered by action: when something about a product changes, all users who follow that product will receive notification.

so far i have 3 schemas for notifications: userNotification, globalNotifications and notificationContent (which I want to share among global and userNotification)

Ok @Anna_N_A ,

So it sounds like the global notifications should have a collection of its own and every user should periodically or on login , query this collection for unread messages. Then the user can specify onit own object that up until x timestamp all global notifications received/read.

notification.find({timestamp : {$gt : <my last timestamp>}}).sort({timestamp : -1})

On the product based notifications you can have a change stream or something of that nature Getting a product change and its Id and then query user collection to any user with an element of this product subscribed.

Then you can have a document per user with the notification written or a bucket of latest x notifications :slight_smile:

Its up to you.

Thanks
Pavel

1 Like

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.